Hierarchical applications

Manifest syntax version 2.0 is focused on reusable component model and delivers a new reactive component model. Improved composability makes it possible to interlink existing applications as parts of a larger setups: Hierarchical Applications.

Using Hierarchical Applications, the two example applications below can be tested separately and later integrated as one. The sample case is a database (possibly shared by multiple systems) and a web front-end (which can operate on either production or test data, but only cares about the resulting connection string).

The examples rely on a workflow.Instance component that serves the purpose of closing the gap between earlier manifest syntax and version 2.0.

Example

The main application reuses the existing database application as part of its hierarchical manifest.

Note

Before using workflow.Instance, be sure to add a Workflow Service to the environment.

{"provisionVms.quantity": 1}

The Workflow Service must be bound to workflow.Instance components. Please refer to the manifest examples below.

Database

application:
  configuration:
    input.app-fork:                      qubell
    input.app-branch:                    HEAD
  interfaces:
    input:
      app-fork: "bind(workflow#input.app-fork)"
      app-branch: "bind(workflow#input.app-branch)"
    output:
      db-hosts: "bind(workflow#result.db-hosts)"
  components:
    workflow:
      type: workflow.Instance
      interfaces:
        input:   # these are the input parameters that will be available to all workflows within the component
          app-fork: configuration(string)
          app-branch: configuration(string)
        result:  # these are the return values that will be visible on the UI and available for using by other components
          db-hosts: publish-signal(list<string>)
      configuration:
        # This is the place to put your workflows, note that launch parameters have been moved to the input interface and
        # configuration
        configuration.workflows:
          launch:
            steps:
              - db-tier:
                  action: .compute.grow
                  parameters:
                    roleName: db-node
                  output:
                    db-hosts: ips
              - deploy-db:
                  action: .deploy.db
                  precedingPhases: [db-tier]
                  parameters:
                    app-fork: "{$.app-fork}"
                    app-branch: "{$.app-branch}"
            return:
              db-hosts:
                value: ${db-hosts}
          .compute.grow:
            parameters:
              - roleName:
                  default: defaultRole
              - quantity:
                  default: 1
              - identity:
                  default: undefined
              - credential:
                  default: undefined
            steps:
              provision-vms:
                action: provisionVms
                parameters:
                  roleName: "{$.roleName}"
                  hardwareId: m1.small
                  quantity: "{$.quantity}"
                  retryCount: 1
                  jcloudsProvider: aws-ec2
                  jcloudsEndpoint: https://ec2.us-east-1.amazonaws.com
                  jcloudsRegions: us-east-1
                  jcloudsNodeNamePrefix: petclinic
                  jcloudsIdentity: ${identity}
                  jcloudsCredential: ${credential}
                  vmIdentity: ubuntu
                  imageId: us-east-1/ami-967edcff
                  securityGroup: default
                output:
                  hosts: ips
            return:
              ips:
                value: ${hosts}
          .deploy.db:
            parameters:
              - app-fork:
                  default: undefined
              - app-branch:
                  default: undefined
            steps:
              - install-db:
                  action: chefrun
                  parameters:
                    roles: [ db-node ]
                    runList: ["recipe[mysql::server]"]
                    isSolo: true
                    recipeUrl: "https://github.com/qubell/cookbooks/raw/0.2/cookbooks.tar.gz"
                    jattrs:
                      mysql:
                        server_root_password: e653e94ee8d064ec95ef5a1381c87a23
                        server_repl_password: e653e94ee8d064ec95ef5a1381c87a23
                        server_debian_password: e653e94ee8d064ec95ef5a1381c87a23
              - deploy-db:
                  action: chefrun
                  precedingPhases: [ install-db ]
                  parameters:
                    roles: [ db-node ]
                    runList: ["recipe[qubell::mysql]"]
                    isSolo: true
                    recipeUrl: "https://github.com/qubell/cookbooks/raw/0.2/cookbooks.tar.gz"
                    jattrs:
                        database:
                          name: "petclinic"
                          user: "jpetstore"
                          password: "jpetstore"
                          schema: "https://raw.github.com/{$.app-fork}/starter-java-web/{$.app-branch}/src/main/resources/db/mysql/petclinic-mysql-schema.sql"
                          data:   "https://raw.github.com/{$.app-fork}/starter-java-web/{$.app-branch}/src/main/resources/db/mysql/petclinic-mysql-dataload.sql"
          destroy:
            steps:
                - destroy:
                    action: undeployEnv

Main Application

application:

  configuration:
    input.db-app-fork:                      qubell
    input.db-app-branch:                    HEAD
    input.web-app-fork:                     qubell
    input.web-app-branch:                   HEAD
    input.app-tier-size:                    "1"

  bindings:
    - [web.workflow, db]

  interfaces:
    manage:
      scale-up:   bind(web.workflow#actions.scale-up)
      scale-down: bind(web.workflow#actions.scale-down)
      update:     bind(web.workflow#actions.update)
    endpoint:
      url:        bind(web.workflow#result.url)
      ha:         bind(web.workflow#result.haproxy-url)
    server:
      db-hosts:  bind(db#output.db-hosts)
      app-hosts: bind(web.workflow#result.app-hosts)
      lb-hosts:  bind(web.workflow#result.lb-hosts)
    input:
      db-app-fork:      bind(db#input.app-fork)
      db-app-branch:    bind(db#input.app-branch)
      web-app-fork:     bind(web.workflow#input.app-fork)
      web-app-branch:   bind(web.workflow#input.app-branch)
      app-tier-size:    bind(web.workflow#input.app-tier-size)


  components:

    db:
      type: reference.Submodule
      configuration:
        __locator.application-id: "hierarchical-db"

      interfaces:
        input:
          app-fork: configuration(string)
          app-branch: configuration(string)
        output:
          db-hosts: publish-signal(list<string>)

    #FRONTEND COMPOSITE
    web:
      components:
        workflow:
          type: workflow.Instance
          interfaces:
            input:   # these are the input parameters that will be available to all workflows within the component
              app-fork:      configuration(string)
              app-branch:    configuration(string)
              app-tier-size: configuration(string)
            db-input:
              db-hosts: consume-signal(list<string>)
            result:  # these are the return values that will be visible on the UI and available for using by other components
              app-hosts: publish-signal(list<string>)
              lb-hosts:  publish-signal(list<string>)
              url: publish-signal(string)
              haproxy-url: publish-signal(string)
            actions:
              scale-up:      receive-command(int app-tier-size, string app-fork, string app-branch => list<string> app-hosts)
              scale-down:    receive-command(int app-tier-size => list<string> app-hosts)
              update:        receive-command(string app-fork, string app-branch)
          required: [db-input]
          configuration:
            configuration.workflows:
              launch:
                steps:
                  #environment properties are required to get external db-host
                  - get-env-props: &getEnvProps
                      action: getEnvironmentProperties
                      output:
                        environment: result
                  - growing-app: &growApp
                      action: .compute.grow
                      parameters:
                          roleName: app-node
                          quantity: "{$.app-tier-size}"
                      output:
                          app-hosts: ips
                  - growing-lb:
                      action: .compute.grow
                      parameters:
                          roleName: lb-node
                      output:
                          lb-hosts: ips
                  - install-lb:
                      action: chefrun
                      precedingPhases: [ growing-lb ]
                      parameters:
                        roles: [ lb-node ]
                        runList: ["recipe[haproxy]"]
                        isSolo: true
                        recipeUrl: "https://github.com/qubell/cookbooks/raw/0.2/cookbooks.tar.gz"
                  - deploy-app: &appInstall
                      action: .install.app
                      precedingPhases: [ growing-lb, growing-app, get-env-props ]
                      parameters:
                        db-hosts: "{$.environment.db-input.db-hosts}"
                        app-hosts: "{$.app-hosts}"
                        app-fork: "{$.app-fork}"
                        app-branch: "{$.app-branch}"
                return:
                  url:
                    description: Url to the application
                    value: "http://{$.lb-hosts[0]}" #check $.lb-tier~hosts[0]
                  haproxy-url:
                    description: Url to haproxy stats
                    value: "http://{$.lb-tier~hosts[0]}:22002/"
                  app-hosts: &returnAppHosts
                    value: ${app-hosts}
                  lb-hosts:
                    value: ${lb-hosts}


              # Scale up stole some steps from, Launch.
              # Note: Not-using this as macro in launch, allows speed-up provisioning.
              scale-up:

                    steps:
                        - get-env-props: *getEnvProps
                        - app-tier: *growApp
                        - app: *appInstall

                    return:
                        app-hosts: *returnAppHosts

              scale-down:
                    steps:
                        - shrink-app-tier:
                            action: destroyVms
                            parameters:
                              roleName: app-node
                              quantity: "{$.app-tier-size}"

                        - setup-lb:
                            action: .setup.lb
                            precedingPhases: [ shrink-app-tier ]
                            parameters:
                               app-hosts: "{$.app-hosts}"

                    return:
                        app-hosts: *returnAppHosts


              update:

                    steps:
                        - get-env-props: *getEnvProps
                        - update-app:
                            action: .deploy.app
                            precedingPhases: [ get-env-props ]
                            parameters:
                              db-hosts: "{$.environment.db-input.db-hosts}"
                              app-fork: "{$.app-fork}"
                              app-branch: "{$.app-branch}"

              .compute.grow:
                parameters:
                    - roleName:
                        default: defaultRole
                    - quantity:
                        default: 1
                    - identity:
                        default: undefined
                    - credential:
                        default: undefined
                steps:
                  provision-vms:
                    action: provisionVms
                    parameters:
                       roleName: "{$.roleName}"
                       hardwareId: m1.small
                       quantity: "{$.quantity}"
                       retryCount: 1
                       jcloudsProvider: aws-ec2
                       jcloudsEndpoint: https://ec2.us-east-1.amazonaws.com
                       jcloudsRegions: us-east-1
                       jcloudsNodeNamePrefix: petclinic
                       jcloudsIdentity: ${identity}
                       jcloudsCredential: ${credential}
                       vmIdentity: ubuntu
                       imageId: us-east-1/ami-967edcff
                       securityGroup: default
                    output:
                       hosts: ips
                return:
                  ips:
                    value: ${hosts}


              destroy:
                steps:
                  - destroy:
                      action: undeployEnv

              .deploy.app:
                  parameters:
                    - db-hosts:
                        description: Database IP address
                    - app-fork:
                        description: Application fork name
                    - app-branch:
                        description: Branch within the application fork
                  steps:
                    - deploy-app:
                        action: chefrun
                        parameters:
                          roles: [ app-node ]
                          runList: ["recipe[qubell::webapp]"]
                          isSolo: true
                          recipeUrl: "https://github.com/qubell/cookbooks/raw/0.2/cookbooks.tar.gz"
                          jattrs:
                            scm:
                              provider: "git"
                              repository: git://github.com/{$.app-fork}/starter-java-web.git
                              revision: "{$.app-branch}"
                            database:
                              name: "petclinic"
                              host: "{$.db-hosts}"
                              user: "petclinic"
                              password: "petclinic"

              .setup.lb:
                  parameters:
                    - app-hosts:
                        description: Application IP address
                  steps:
                    - setup-lb:
                        action: chefrun
                        parameters:
                          roles: [ lb-node ]
                          runList: ["recipe[qubell::lb]"]
                          isSolo: true
                          recipeUrl: "https://github.com/qubell/cookbooks/raw/0.2/cookbooks.tar.gz"
                          jattrs:
                            haproxy.rebalance:
                              nodes: "{$.app-hosts}"

              .install.app:
                  parameters:
                    - db-hosts:
                        description: Database IP address
                    - app-hosts:
                        description: Application IP address
                    - app-fork:
                        description: Application fork name
                    - app-branch:
                        description: Branch within the application fork
                  steps:
                    - install-app:
                        action: chefrun
                        parameters:
                          roles: [ app-node ]
                          runList: ["recipe[tomcat]"]
                          isSolo: true
                          recipeUrl: "https://github.com/qubell/cookbooks/raw/0.2/cookbooks.tar.gz"

                    - deploy-app:
                        action: .deploy.app
                        precedingPhases: [ install-app ]
                        parameters:
                          db-hosts: "{$.db-hosts}"
                          app-fork: "{$.app-fork}"
                          app-branch: "{$.app-branch}"

                    - setup-lb:
                        action: .setup.lb
                        precedingPhases: [ deploy-app ]
                        parameters:
                          app-hosts: "{$.app-hosts}"

Markers and Environment Properties

Markers and environment properties must be referenced explicitly. The example below demonstrates how to link an application to environment services.

application:
  interfaces:
    output:
      red: "bind(workflow#result.red)"
      green: "bind(workflow#result.green)"
  components:
    workflow:
      type: workflow.Instance
      interfaces:
        result:
          red: publish-signal(string)
          green: publish-signal(int)
        has-internet-access:
          has-internet-access: consume-signal(unit)
        properties:
          sample-property-red: consume-signal(string)
          sample-property-green: consume-signal(int)
      required: [has-internet-access, properties]
      configuration:
        configuration.workflows:
          launch:
            steps:
              - get-env-props:
                  action: getEnvironmentProperties
                  output:
                    props: result
            return:
              red:
                value: "{$.props.properties.sample-property-red}"
              green:
                value: "{$.props.properties.sample-property-green}"
          destroy:
            steps: []
    marker:
      type: reference.Service
      interfaces:
        has-internet-access:
          has-internet-access: publish-signal(unit)
    props:
      type: reference.Service
      interfaces:
        properties:
          sample-property-red: publish-signal(string)
          sample-property-green: publish-signal(int)
  bindings:
      - [workflow, marker]
      - [workflow, props]