Software Deployment

Agenda

  • Introduction

  • Software Packaging

  • Execution Environments

  • Packaging Solutions

  • Package Publication

Introduction

Definition

Software deployment is all of the activities that make a software system available for use.
— Pressman

Deployment

software deployment
Figure 1. The Big Picture

Activities

deployment activities
Figure 2. Deployment Activities

Release

The release activity follows from the completed development process, and is sometimes classified as part of the development process rather than deployment process. It includes all the operations to prepare a system for assembly and transfer to the computer system(s) on which it will be run in production. Therefore, it sometimes involves determining the resources required for the system to operate with tolerable performance and planning and/or documenting subsequent activities of the deployment process.

Installation and activation (1/4)

For simple systems, installation involves establishing some form of command, shortcut, script or service for executing the software (manually or automatically). For complex systems it may involve configuration of the system – possibly by asking the enduser questions about its intended use, or directly asking them how they would like it to be configured – and/or making all the required subsystems ready to use. Activation is the activity of starting up the executable component of software for the first time (not to be confused with the common use of the term activation concerning a software license, which is a function of Digital Rights Management systems.)

Installation and activation (2/4)

In larger software deployments on servers, the main copy of the software to be used by users - "production" - might be installed on a production server in a production environment. Other versions of the deployed software may be installed in a test environment, development environment and disaster recovery environment.

Installation and activation (3/4)

In complex continuous delivery environments and/or software as a service systems, differently-configured versions of the system might even exist simultaneously in the production environment for different internal or external customers (this is known as a multi-tenant architecture), or even be gradually rolled out in parallel to different groups of customers, with the possibility of cancelling one or more of the parallel deployments.

Installation and activation (4/4)

For example, Twitter is known to use the latter approach for A/B testing of new features and user interface changes. A "hidden live" group can also be created within a production environment, consisting of servers that are not yet connected to the production load balancer, for the purposes of blue-green deployment.

Deactivation

Deactivation is the inverse of activation, and refers to shutting down any already-executing components of a system. Deactivation is often required to perform other deployment activities, e.g., a software system may need to be deactivated before an update can be performed. The practice of removing infrequently used or obsolete systems from service is often referred to as application retirement or application decommissioning.

Uninstallation

Uninstallation is the inverse of installation. It is the removal of a system that is no longer required. It may also involve some reconfiguration of other software systems in order to remove the uninstalled system’s dependencies.

Update

The update process replaces an earlier version of all or part of a software system with a newer release. It commonly consists of deactivation followed by installation. On some systems, such as on Linux when using the system’s package manager, the old version of a software application is typically also uninstalled as an automatic part of the process. (This is because Linux package managers do not typically support installing multiple versions of a software application at the same time, unless the software package has been specifically designed to work around this limitation.)

Built-in update

Mechanisms for installing updates are built into some software systems (or, in the case of some operating systems such as Linux, Android and iOS, into the operating system itself). Automation of these update processes ranges from fully automatic to user initiated and controlled. Norton Internet Security is an example of a system with a semi-automatic method for retrieving and installing updates to both the antivirus definitions and other components of the system. Other software products provide query mechanisms for determining when updates are available.

Version tracking (1/2)

Version tracking systems help the user find and install updates to software systems. For example: Software Catalog stores version and other information for each software package installed on a local system. One click of a button launches a browser window to the upgrade web page for the application, including auto-filling of the user name and password for sites that require a login.

Version tracking (2/2)

On Linux, Android and iOS this process is even easier because a standardized process for version tracking (for software packages installed in the officially supported way) is built into the operating system, so no separate login, download and execute steps are required – so the process can be configured to be fully automated. Some third-party software also supports automated version tracking and upgrading for certain Windows software packages.

Software Packaging

software packaging
  • Packaging is the process of putting together software artifacts: distributables, executables, documentation, libraries, resources, etc.

  • In short, everything that is needed to deploy, install and execute a piece of software.

  • A Software Package can be a jar archive, a Debian package file, a self-extracting installer, an Android Package (APK), etc.

Packages and Artifacts

Software Artifacts

Artifacts may be composed of and may depend on other artifacts.

cd software packages
Figure 3. Packages and Artifacts

Software Package Example: NeoEMF

neoemf artifacts

Artifact Dependencies

neoemf core artifacts

Dependencies at Different Moments

different dependencies

Optional Dependencies

optional dependencies

Transitive Dependencies

transitive dependencies

Provided Dependencies

provided dependencies

Some dependencies may be provided by the execution environment (application container).

Execution Environments

Environment

  • The execution environment is the platform where packages are deployed and executed

  • OS, Application containers, Web containers, virtual images, etc.

What is a Container?

  • Operating system–level virtualization

  • Operating system–level virtualization is a server virtualization method where the kernel of an operating system allows for multiple isolated user-space instances, instead of just one

  • Differs from traditional virtual machines

Containers

containers vs vm
Figure 4. Containers vs Virtual Machines

Examples

Generic Containers
  1. Jails

  2. Docker

  3. Podman

Java Containers
  1. Standalone

  2. Maven Artifacts

  3. OSGI

    1. Eclipse Equinox

    2. Apache Karaf

OSGI Containers

osgi logo
  • OSGI is a standard that specifies a framework for managing the lifecycle of Java applications.

apache karaf
Figure 5. Apache Karaf

OSGI Bundles

osgi bundle lifecycle
Figure 6. OSGI Bundle Lifecycle

Docker

docker logo
  • Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application

  • Makes fast running, portable containers

Packaging Solutions

Popular Solutions

Java
  1. Standalone

  2. Maven Artifacts

  3. OSGI

    1. Eclipse Equinox

    2. Apache Karaf

Containers
  1. Jails

  2. Docker

  3. Podman

Java Standalone Applications

  • All-in-one Jar

Maven Artifact Packaging

OSGI Packaging (Bundles)

  • Jar archive + Manifest.MF

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Sample
Bundle-SymbolicName: com.exoscale.sample
Bundle-Version: 1.0.0.qualifier
Bundle-Activator: com.exoscale.sample.Activator
Bundle-Vendor: Exoscale
Require-Bundle: com.exoscale.otherbundle
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Service-Component: OSGI-INF/service.xml
Import-Package: com.exoscale.services.example;version="3.0.0“
Export-Package: com.sample.utils

Manifest Example

Manifest.MF
Manifest-Version: 1.0
Bnd-LastModified: 1571834388995
Build-Jdk: 1.8.0_202
Built-By: sunye
Bundle-ActivationPolicy: lazy
Bundle-ClassPath: .,chronicle-map-3.17.1.jar,chronicle-core-2.17.6.jar,c
 hronicle-values-2.16.1.jar,javapoet-1.5.1.jar,chronicle-threads-2.17.2.
 jar,affinity-3.1.11.jar,slf4j-api-1.7.25.jar,chronicle-wire-2.17.26.jar
 ,compiler-2.3.4.jar,chronicle-algorithms-1.16.0.jar,jna-4.2.1.jar,jna-p
 latform-4.2.1.jar,chronicle-bytes-2.17.19.jar,cglib-nodep-3.2.10.jar
Bundle-Description: NeoEMF core component.        Specialize the standar
 d EMF classes and provide abstract classes for dedicated back-end exten
 sions.
Bundle-DocURL: http://www.atlanmod.org
Bundle-License: https://www.eclipse.org/legal/epl-2.0/
Bundle-ManifestVersion: 2
Bundle-Name: NeoEMF Core
Bundle-SymbolicName: fr.inria.atlanmod.neoemf.core
Bundle-Vendor: Atlanmod
Bundle-Version: 2.0.1.20191023123732
Created-By: Apache Maven Bundle Plugin
Embed-Dependency: *;scope=compile|runtime;type=!pom;groupId=!fr.inria.at
 lanmod.neoemf|org.atlanmod.commons|org.eclipse.emf|org.apache.logging.l
 og4j
Embed-Transitive: true
Embedded-Artifacts: chronicle-map-3.17.1.jar;g="net.openhft";a="chronicl
 e-map";v="3.17.1",chronicle-core-2.17.6.jar;g="net.openhft";a="chronicl
 e-core";v="2.17.6",chronicle-values-2.16.1.jar;g="net.openhft";a="chron
 icle-values";v="2.16.1",javapoet-1.5.1.jar;g="com.squareup";a="javapoet
 ";v="1.5.1",chronicle-threads-2.17.2.jar;g="net.openhft";a="chronicle-t
 hreads";v="2.17.2",affinity-3.1.11.jar;g="net.openhft";a="affinity";v="
 3.1.11",slf4j-api-1.7.25.jar;g="org.slf4j";a="slf4j-api";v="1.7.25",chr
 onicle-wire-2.17.26.jar;g="net.openhft";a="chronicle-wire";v="2.17.26",
 compiler-2.3.4.jar;g="net.openhft";a="compiler";v="2.3.4",chronicle-alg
 orithms-1.16.0.jar;g="net.openhft";a="chronicle-algorithms";v="1.16.0",
 jna-4.2.1.jar;g="net.java.dev.jna";a="jna";v="4.2.1",jna-platform-4.2.1
 .jar;g="net.java.dev.jna";a="jna-platform";v="4.2.1",chronicle-bytes-2.
 17.19.jar;g="net.openhft";a="chronicle-bytes";v="2.17.19",cglib-nodep-3
 .2.10.jar;g="cglib";a="cglib-nodep";v="3.2.10"
Export-Package: fr.inria.atlanmod.neoemf.bind;version="2.0.1";uses:="fr.
 inria.atlanmod.neoemf.data",fr.inria.atlanmod.neoemf.config;version="2.
 0.1";uses:="fr.inria.atlanmod.neoemf.data.store,fr.inria.atlanmod.neoem
 f.data.store.listener",fr.inria.atlanmod.neoemf.core;version="2.0.1";us
 es:="fr.inria.atlanmod.neoemf.data.store,fr.inria.atlanmod.neoemf.data.
 store.adapter",fr.inria.atlanmod.neoemf.data;version="2.0.1";uses:="fr.
 inria.atlanmod.neoemf.config,fr.inria.atlanmod.neoemf.core,fr.inria.atl
 anmod.neoemf.data.bean,fr.inria.atlanmod.neoemf.data.mapping",fr.inria.
 atlanmod.neoemf.data.bean;version="2.0.1";uses:="fr.inria.atlanmod.neoe
 mf.core",fr.inria.atlanmod.neoemf.data.bean.serializer;version="2.0.1";
 uses:="fr.inria.atlanmod.neoemf.core,fr.inria.atlanmod.neoemf.data.bean
 ",fr.inria.atlanmod.neoemf.data.mapping;version="2.0.1";uses:="fr.inria
 .atlanmod.neoemf.core,fr.inria.atlanmod.neoemf.data,fr.inria.atlanmod.n
 eoemf.data.bean",fr.inria.atlanmod.neoemf.data.store;version="2.0.1";us
 es:="fr.inria.atlanmod.neoemf.config,fr.inria.atlanmod.neoemf.core,fr.i
 nria.atlanmod.neoemf.data,fr.inria.atlanmod.neoemf.data.bean,fr.inria.a
 tlanmod.neoemf.data.mapping,fr.inria.atlanmod.neoemf.data.store.adapter
 ,fr.inria.atlanmod.neoemf.data.store.listener",fr.inria.atlanmod.neoemf
 .data.store.adapter;version="2.0.1";uses:="fr.inria.atlanmod.neoemf.cor
 e,fr.inria.atlanmod.neoemf.data,fr.inria.atlanmod.neoemf.data.store,fr.
 inria.atlanmod.neoemf.resource",fr.inria.atlanmod.neoemf.data.store.lis
 tener;version="2.0.1";uses:="fr.inria.atlanmod.neoemf.data",fr.inria.at
 lanmod.neoemf.resource;version="2.0.1";uses:="fr.inria.atlanmod.neoemf.
 config,fr.inria.atlanmod.neoemf.core,fr.inria.atlanmod.neoemf.data.stor
 e,fr.inria.atlanmod.neoemf.data.store.adapter",fr.inria.atlanmod.neoemf
 .util;version="2.0.1",fr.inria.atlanmod.neoemf.util.compare;version="2.
 0.1";uses:="com.google.common.cache"
Import-Package: org.eclipse.emf.compare;resolution:=optional,com.google.
 common.cache;resolution:=optional;version="[21.0,22)"
Provide-Capability: osgi.service;objectClass:List<String>="fr.inria.atla
 nmod.neoemf.config.Config",osgi.service;objectClass:List<String>="fr.in
 ria.atlanmod.neoemf.data.BackendFactory",osgi.service;objectClass:List<
 String>="fr.inria.atlanmod.neoemf.util.UriFactory"
Require-Bundle: org.atlanmod.commons;visibility:=reexport,org.eclipse.em
 f.ecore;visibility:=reexport
Require-Capability: osgi.extender;filter:="(&(osgi.extender=osgi.compone
 nt)(version>=1.3.0)(!(version>=2.0.0)))",osgi.ee;filter:="(&(osgi.ee=Ja
 vaSE)(version=1.8))"
Service-Component: OSGI-INF/fr.inria.atlanmod.neoemf.data.im.InMemoryBac
 kendFactory.xml,OSGI-INF/fr.inria.atlanmod.neoemf.data.im.config.InMemo
 ryConfig.xml,OSGI-INF/fr.inria.atlanmod.neoemf.data.im.util.InMemoryUri
 Factory.xml
Tool: Bnd-4.1.0.201810181252

Docker

docker container
Layers
  • Images are based on a parent

  • The layers stack on top

  • Files in base layers are shared between Images

  • Each commit creates a layer

  • Base image has no parent

Docker Packaging (Docker file)

Docker file for running NeoEMF’s benchmarks
FROM maven:alpine

ENV NEOEMF_HOME /root/ws
VOLUME $NEOEMF_HOME

# Build project & benchmarks
WORKDIR /tmp/neoemf
COPY . .
RUN mvn -B install -DskipTests -pl !neoemf-tests  \
 && mvn -B package -DskipTests -f benchmarks/pom.xml -Duberjar.directory=/root/ \
 && rm -rf /root/.m2/* /tmp/*

# Install JProfiler agent
RUN wget https://download-keycdn.ej-technologies.com/jprofiler/jprofiler_linux_10_1_5.tar.gz -P /tmp/ \
 && tar -xzf /tmp/jprofiler_linux_10_1_5.tar.gz -C /usr/local \
 && mv /usr/local/jprofiler10.1.5 /usr/local/jprofiler \
 && rm /tmp/jprofiler_linux_10_1_5.tar.gz

EXPOSE 8849

# Configure container
WORKDIR /root
CMD ["-help"]
ENTRYPOINT ["java", "-jar", "neoemf-benchmarks.jar"]

Package Publication

Maven Artifact Publication

deploy maven central
  1. Artifact upload mvn deploy

  2. Staging repository at a Repository Manager (e.g. Nexus)

  3. Publication on the Repository (e.g. NeoEMF-Core)

OSGI Artifact Publication

osgi services layer
Figure 7. OSGI Services Layer

Docker Artifact Publication

docker hub
Figure 8. Publication on Docker Hub

Conclusion