ingenieux docs

Using Arbitrary Platforms

We've made it!

This is something we've been looking for years, and while not directly related to our plugin, we're sure others would love to learn from it: How to use AWS Elastic Beanstalk using a Platform NOT supported by AWS EB Stacks.

While there were some interesting ideas - like this one from Jetty -, they're more erosion-prone

The idea

Write an app to wrap the execution for another app, using the tradicional exec(2) syscall.

Among all stacks available for AWS EB, Node.js is the easiest to hack, due to its simplicity and self-contained approach. The closest one would be python, but as it is locked up to 2.6 (as of Mar 2013), it would offer compatibility challenges, so here's a rundown of what you'd need:

Write a self-contained launcher

Under AWS EB, the Node.js stack requires a main js file and a package.json (for NPM). Due to the NPM nature, it is also quite worthy to have a npm-shrinkwrap file. Lets quote the docs

Node Command–Lets you enter the command used to start the Node.js application. An empty string (the default) means AWS Elastic Beanstalk will use app.js, then server.js, and then "npm start" in that order.

So lets try a bare bones server.js suitable for Jetty Runner. All we need is to make server.js wrap the command line for jetty runner and launch that as an exec call (made using node-kexec). So here's our launcher for jetty-runner:

#!/usr/bin/env node

var os = require("os"),
    globule = require("globule"),
    kexec = require("kexec"),
    util = require("util");

JAVA_BIN = "/usr/bin/java"
BASEDIR = __dirname;

JETTY_RUNNER_JAR = function() { 
  return globule.find('WEB-INF/lib/jetty-runner*.jar', { srcBase: BASEDIR })[0];
}();

JETTY_PORT = process.env['PORT'] || 8080;

var args = [ "-jar", JETTY_RUNNER_JAR, "--port", JETTY_PORT, BASEDIR ]

console.log(util.format("Running %s on port %s with args: %s", JAVA_BIN, JETTY_PORT, args));

kexec(JAVA_BIN, args);

Here's how it works:

  • picks up the current directory name
  • finds the jetty-runner jar being used on WEB-INF/lib
  • and PORT to be bound into
  • builds a CLI
  • does a exec out of it

You'll also need a package.json file:

{
  "name": "webapp",
  "version": "0.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node server.js"
  },
  "dependencies": {
    "kexec": "0.2.0",
    "globule" : "0.2.0"
  }
}

Now what?

Now you need to have a zip to deploy into AWS EB. It expects the base stack requirements (the .js and json file), as well as the .ebextensions if needed:

  • server.js and package.json
  • the .ebextensions subdirectory

Once in AWS, all you need is to create a Node.js application under AWS EB, and upload this zip as a new application version. The process output will be kept in node.js, which is good, for, say, snapshot logs.

Testing it

You don't need to deploy to test if its working fine. Build your package, unzip into a directory (or mvn package in the example at the bottom), cd to the package source and just run the launcher:

aldrin@vault standalone-awseb-app$ cd target/standalone-awseb-app-0.0.1-SNAPSHOT
aldrin@vault standalone-awseb-app-0.0.1-SNAPSHOT$ npm install && npm shrinkwrap && node server.js
npm WARN package.json webapp@0.0.0 No description
npm WARN package.json webapp@0.0.0 No repository field.
npm WARN package.json webapp@0.0.0 No README data
npm http GET https://registry.npmjs.org/kexec/-/kexec-0.2.0.tgz
npm http GET https://registry.npmjs.org/globule/-/globule-0.2.0.tgz
npm http 200 https://registry.npmjs.org/kexec/-/kexec-0.2.0.tgz
npm http 200 https://registry.npmjs.org/globule/-/globule-0.2.0.tgz

> kexec@0.2.0 install /Users/aldrin/projetos/sources/standalone-awseb-app/target/standalone-awseb-app-0.0.1-SNAPSHOT/node_modules/kexec
> node-gyp configure build

  CXX(target) Release/obj.target/kexec/src/kexec.o
  SOLINK_MODULE(target) Release/kexec.node
  SOLINK_MODULE(target) Release/kexec.node: Finished
npm http GET https://registry.npmjs.org/lodash/-/lodash-2.4.1.tgz
npm http GET https://registry.npmjs.org/glob/-/glob-3.2.9.tgz
npm http GET https://registry.npmjs.org/minimatch/-/minimatch-0.2.14.tgz
npm http 200 https://registry.npmjs.org/lodash/-/lodash-2.4.1.tgz
npm http 200 https://registry.npmjs.org/minimatch/-/minimatch-0.2.14.tgz
npm http 200 https://registry.npmjs.org/glob/-/glob-3.2.9.tgz
npm http GET https://registry.npmjs.org/lru-cache/-/lru-cache-2.5.0.tgz
npm http GET https://registry.npmjs.org/sigmund/-/sigmund-1.0.0.tgz
npm http GET https://registry.npmjs.org/inherits/-/inherits-2.0.1.tgz
npm http 200 https://registry.npmjs.org/sigmund/-/sigmund-1.0.0.tgz
npm http 200 https://registry.npmjs.org/lru-cache/-/lru-cache-2.5.0.tgz
npm http 200 https://registry.npmjs.org/inherits/-/inherits-2.0.1.tgz
globule@0.2.0 node_modules/globule
├── glob@3.2.9 (inherits@2.0.1)
├── minimatch@0.2.14 (sigmund@1.0.0, lru-cache@2.5.0)
└── lodash@2.4.1

kexec@0.2.0 node_modules/kexec
wrote npm-shrinkwrap.json
Running /usr/bin/java on port 8080 with args: -jar,WEB-INF/lib/jetty-runner-9.1.3.v20140225.jar,--port,8080,/Users/aldrin/projetos/sources/standalone-awseb-app/target/standalone-awseb-app-0.0.1-SNAPSHOT
2014-03-13 03:21:27.792:INFO::main: Logging initialized @78ms
2014-03-13 03:21:27.798:INFO:oejr.Runner:main: Runner
2014-03-13 03:21:27.887:INFO:oejs.Server:main: jetty-9.1.3.v20140225
2014-03-13 03:21:31.185:INFO:oeja.AnnotationConfiguration:main: Scanned 1 container path jars, 60 WEB-INF/lib jars, 1 WEB-INF/classes dirs in 2852ms for context o.e.j.w.WebAppContext@6416298a{/,file:/Users/aldrin/projetos/sources/standalone-awseb-app/target/standalone-awseb-app-0.0.1-SNAPSHOT/,STARTING}{file:/Users/aldrin/projetos/sources/standalone-awseb-app/target/standalone-awseb-app-0.0.1-SNAPSHOT/}
03:21:31,233 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
03:21:31,233 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
03:21:31,233 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/aldrin/projetos/sources/standalone-awseb-app/target/standalone-awseb-app-0.0.1-SNAPSHOT/WEB-INF/classes/logback.xml]
03:21:31,273 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
03:21:31,278 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
03:21:31,287 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
03:21:31,335 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - This appender no longer admits a layout as a sub-component, set an encoder instead.
03:21:31,335 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder.
03:21:31,335 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - See also http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details
03:21:31,336 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.eclipse.jetty.util.log] to WARN
03:21:31,336 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.apache] to WARN
03:21:31,336 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
03:21:31,336 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
03:21:31,336 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
03:21:31,338 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7ed32b58 - Registering current configuration as safe fallback point

But I use beanstalker

Amazing! All you need is to tweak your pom. Here's what you'll need:

  • Create those files under src/main/webapp path so it gets on webapp root.
  • Make sure you've got jetty-runner under WEB-INF/lib (it will be ignored by other appservers class loaders so it looks quite safe):
<dependency>
  <groupId>org.eclipse.jetty</groupId>
  <artifactId>jetty-runner</artifactId>
  <version>9.1.3.v20140225</version>
</dependency>
  • Change beanstalk.solutionStack to 64bit Amazon Linux 2013.09 running Node.js
  • Make sure your .ebextensions match the runtime requirements. E.g., since you're running java on a Node.js-based stack, you'll need something like this under .ebextensions:
packages:
  yum:
    java-1.7.0-openjdk: []
    java-1.7.0-openjdk-devel: []
commands:
  use_java7:
    command: alternatives --set java /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java

A Full Example

The jetty launcher example we've used is hosted on bitbucket on this URL: https://bitbucket.org/aldrinleal/standalone-awseb-app. Once you clone it and revise the cnamePrefix and environmentRef, just run this command:

$ mvn -Pfast-deploy beanstalk:create-environment

Notice we removed tests from the original and tweaked the jetty deps a little, in order to accomodate the jetty.version bump from 7 to 9

Hope it helps!

© ingenieux Labs 2013 | Mixed with Bootstrap v2.3.1 | Baked with JBake v2.2.1

comments powered by Disqus