Friday, December 19, 2014

Amazon Web Services Developer Flight Checklist


Working in the clouds on Amazon AWS benefits from some best practices.  I am capturing details into a pre-flight checklist before you fly between the clouds as I come across them in my daily development.

Amazon Web Service Best Practices

Elastic Beanstalk

1) Add JVM tracking like New Relic
2) Add Log tracking like SumoLogic or Splunk
3) Add GEOIP tracking like MaxMind geoIP
When deploying the mod_geoip conf file to elastic beanstalk remember that we are running behind an elastic load balancer proxy.  This proxy will add its IP address to the %{X-Forwarded-For} variable.  geoIP will not be able to pick out the actual client IP until you add the following parameter to the conf file to workaround this nonstandard NCSA log format.
see http://dev.maxmind.com/geoip/legacy/mod_geoip2/#Proxy-Related_Directives
4) Install an SSL certificate on the ELB (Elastic Load Balancer)
Follow http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/ssl-server-cert.html but gunzip, tar xvf and cd to the install directory first.

EC2 instances need the Java AWS SDK
obrienlabs-mbp15:_deployment michaelobrien$ sudo scp ~/.aws/credentials ec2-user@54.5.6.7:/home/ec2-user
Password:

credentials                                                                                                                            100%  221     0.2KB/s   00:00  

Pick your JVM
They do seem to quietly upgrade the JDK after official published versions - they have since went for 1.6.0_24 to 32 and 1.7.0_51 to 72
OpenJDK versions lag SUN by about a week, but amazon seems to favour Java 8
The 3 ami versions for 6, 7 and 8 look to be behind more as we drop JDK versions - with Java 8 up to date and 6 way behind.

OpenJDK versions
Beanstalk Tomcat 8 Java 8 =  1.8.0_25 (latest 25) 2014.09 v1.0.0
Beanstalk Tomcat 7 Java 7 = 1.7.0_65 (latest 72) 2014.09 v1.0.9
Beanstalk Tomcat 7 Java 6 = 1.6.0_32 (latest 85) 2014.09 v1.0.9

Their official page on supported platforms
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.platforms.html#concepts.platforms.java
On all your windows machines install cygwin and enable the ssh server.
http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/

AWS Technical Details

Amazon Service Disruptions:

Some AWS services encounter issues from time to time at the service, zone or region.  If you happen to be in the console you see the occasional message at the top - however the RSS feed gives a bit more detail on the issue.  Again if you build in standard zone and region redundancy then failover kicks in and no service disruption affects your instances.

Install the client - passed by my teamlead
https://chrome.google.com/webstore/detail/rss-feed-reader/pnjaodmkngahhkoihejjehlcdlnohgmp?hl=en

VPC restrictions

us-east-1a is restricted by amazon for new subnets (I read this somewhere – and also read that is was full) – and for t2 instances

us-east-1c came back as restricted during a CloudFormation json deployment run – but I read somewhere that this may be only for customers that have accounts predating when the “default” VPC (only 1 public subnet at 172.31.0.0/16) – customers like my personal  that signed up after Aug 2014 get are in the default VPC.

Elastic Beanstalk

Configuring a local OSX dev environment for Beanstalk

see

Get an SSL Certificate signed by Entrust

Entrust will reply to your request within 10 min.

obrienlabs-mbp15:nutridat_domain_cert michaelobrien$ openssl genrsa 2048 > privatekey.pem
Generating RSA private key, 2048 bit long modulus
..+++
.............+++
e is 65537 (0x10001)

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CA
State or Province Name (full name) [Some-State]:Ontario
Locality Name (eg, city) []:Ottawa
Organization Name (eg, company) [Internet Widgits Pty Ltd]:.
Organizational Unit Name (eg, section) []:.
Common Name (e.g. server FQDN or YOUR name) []:nutridat.org
Email Address []:michael@o

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:

An optional company name []:.


Elastic Beanstalk on AWS runs on a customized EC2 AMI as Tomcat 6/7/8 on top of Apache HTTPD.
Configure your maven project to compile against Tomcat 7 or 8

  
    
      junit
      junit
      3.8.1
      test
    
            
        
            jstl
            jstl
            1.2
            
               
        
            javax.el
            el-api
            2.2
            provided
        
                
        
            org.apache.tomcat
            tomcat-el-api
            ${version.tomcat}
            provided
        
        
            org.apache.tomcat
            tomcat-servlet-api
            ${version.tomcat}
            provided
        
  

No automatic maven dependencies in eclipse/sts? add the following
 
  
   
Get the AWS CLI and EC2 CLI
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/set-up-ec2-cli-linux.html#setting_up_ec2_command_linux
curl "http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip" -o "ec2-api-tools.zip"
obrienlabs-mbp15:aws michaelobrien$ ec2-describe-instances
 

http://docs.aws.amazon.com/cli/latest/userguide/installing.html
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
obrienlabs-mbp15:aws michaelobrien$ aws --version
aws-cli/1.7.0 Python/2.7.6 Darwin/14.0.0
Install Apache HTTP Server
Install Apache Tomcat Web Container
When registering a domain with a Canadian address - you will need a space in the postal code to get past the verification check on AWS Route53
EC2 Get metadata using the link local address http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
Increase the upload size for files greater than 5Mb in server.xml by changing the maxSwallowSize

    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               maxSwallowSize="52428800"
               redirectPort="8443" />
Install monitoring services like SumoLogic, New Relic and Boundary

curl -fsS -d '{"token":"api.6f44444444-3333"}' -H 'Content-Type: application/json' https://meter.boundary.com/setup_meter > setup_meter.sh && chmod +x setup_meter.sh && ./setup_meter.sh

Issues
20150408: beanstalk environment configuration update does not work - it fails to update the AMI
before ami: ami-986327f0
after update from 2014.09 v1.2.0 to 2015.03 v1.3.0
after ami: ami-986327f0
2015-04-08 11:45:33 UTC-0400 ERROR Update environment operation is complete, but with errors. For more information, see troubleshooting documentation. 


Since the default ami for 2015.03 v1.3.0 is
ami-0c6f5f64

This should be the ami during the upgrade
Fix: replace the ami with the new ami-0c6f5f64 version after the AWS upgrade
2015-04-08 15:13:12 UTC-0400 INFO Environment health has transitioned from RED to GREEN 

AWS Technical Links

Beanstalk CloudFormation parameters - http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options.html
Elastic Container Service - http://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html
http://aws.amazon.com/blogs/aws/ec2-container-service-in-action/?sc_ichannel=ha&sc_ipage=homepage&sc_icountry=en&sc_isegment=c&sc_iplace=hero1&sc_icampaigntype=product_launch&sc_icampaign=ha_en_ECS_Launch&sc_icategory=none&sc_idetail=ha_en_281_1&sc_icontent=ha_281&
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_GetStarted.html
Send Mail - http://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-using-smtp-java.html SES Sending Limits - https://console.aws.amazon.com/ses/home?region=us-east-1#dashboard: Installing AWS CLI - http://docs.aws.amazon.com/cli/latest/userguide/installing.html

configure secure access
http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_VirtualMFA.html

Get all the hosts up on your private subnet

for ip in $(seq 1 254); do ping -c 1 192.168.0.$ip>/dev/null; [ $? -eq 0 ] && echo "192.168.0.$ip UP" || : ; done

Notes:
20150329: RDS Performance: My t2-micro RDS instances are taking 13 sec for all queries (count, select, filter) where they used to take less than 1.  Rebooting - no change, rebuilding the RDS instance decreases the time to 9 sec (indexing?)
When I switched up to a t2-small with double the ram - speed increased to 6 sec.  Therefore I have hit some sweet spot with my 3 million biometric records.
After a couple warm up queries (directly in MySQL Workbench) to kick in burst mode for "select count(1) from biometric.gps_record"
t2-micro = 11.9 sec
t2-small = 2.5 to 1.1 sec
t2-medium = 1.6 to 0.9 sec 
I took a T2.medium (2 threads & 4G) and modified it back to a T2.micro (1 thread @ 1G) and performance went from 1.1 sec to 10 sec as expected - as the query is memory intensive and peaks at 1700Mb from a baseline of 450Mb - therefore a T2.small (1 thread @ 2G) will sufficed until I run up 6 million records




Thursday, November 6, 2014

Amazon Web Services Cloud PaaS using Elastic Beanstalk on top of Elastic Compute Cloud

In progress - 20141104


Purpose:

    Embrace DevOps by running everything in the cloud - the only thing you need locally is a very good developer machine.

How to become a DevOps developer.
    I need a platform as a service solution for both the application server layer and the database layer but I would like to retain some control over the infrastructure.  There are several solutions including Oracle Cloud, Google AppSpot, Pivotal CloudFoundry, Cloudbees, Heroku and other providers - however there is one service that has almost everything we need - Amazon Web Services.

    I ran Oracle cloud for 6 months - specifically the PaaS offering around their $275/month Java service and their $250/month database service.
http://eclipsejpa.blogspot.ca/2013/06/java-cloud-providers-review.html

However, I found that it did not offer the flexibility that AWS currently offers in a combined Iaas and PaaS model.

Account Setup:

Get an Amazon Web Services account at http://aws.amazon.com
Get Atlassian JIRA, Bitbucket and Bamboo accounts
Link your bamboo and jira accounts https://confluence.atlassian.com/display/BITBUCKET/Linking+Bitbucket+and+GitHub+accounts+to+JIRA
Elastic Beanstalk supports Java 8 and Tomcat 8 but you need to enable Java 8 first https://jira.atlassian.com/browse/BAM-15299#

Developer Setup:

A good machine - ideally the top Mac Pro
VMware Fusion for running multiple OSX, Redhat and Windows virtual machines
MySQL (or PostgreSQL)

Connect to your EC2 or RDS instance

Follow

obrienlabs-mbp15:keys michaelobrien$ chmod 400 obrien_systems_aws_20141115.pem  obrienlabs-mbp15:keys michaelobrien$ ssh-add obrien_systems_aws_20141115.pem obrienlabs-mbp15:keys michaelobrien$ ssh -i obrien_systems_aws_20141115.pem ec2-user@ec2-11-222-333-444.compute-1.amazonaws.com https://aws.amazon.com/amazon-linux-ami/2014.09-release-notes/
No packages needed for security; 1 packages available Run "sudo yum update" to apply all updates. [ec2-user@ip-172-31-01-01 ~]$ 
or
obrienlabs-mbp15:keys michaelobrien$ ssh-add obrien_systems_aws_20141115.pem
to bastion
obrienlabs-mbp15:keys michaelobrien$ ssh -A ec2-user@ec2-11-222-333-444.compute-1.amazonaws.com
then to instance
obrienlabs-mbp15:keys michaelobrien$ ssh ec2-user@ec2-172-20-1-244.compute-1.amazonaws.com

Make sure you are using the latest AWS CLI as some API has been upgraded
[ec2-user@ip-10-0-1-106 ~]$ aws --version aws-cli/1.5.1 Python/2.6.9 Linux/3.14.20-20.44.amzn1.x86_64

Integration:

    You will want Continuous Integration support in the form of a GIT hosting service that supports Maven.   GitHub is one possibility but I find the Atlassian has a rich ecosystem that integrates very well with Amazon AWS.  A bamboo build account can automate building and deploying to your elastic beanstalk EC2 instances with a minimal amount of scripting.



Throughput:

http://java.awsblog.com/post/TxBSZ02ZOG3VPZ/AWS-SDK-for-Java-Maven-Modules

ScaleArc

ScaleArc as a software SQL cache and read/write sharding proxy sees its best performance when the number of threads is a small multiple of the number of cores in the proxy.
http://java.dzone.com/articles/scalearc-benchmarking-sysbench

20141112 AWS re:Invent Keynote


AWS EC2 container service (docker)
https://aws.amazon.com/ecs/
ECS arrived 20141218
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html

Google just came out with an alpha version of their microservices container – a 3rd option to look at briefly.  


High Availability live key rotation and tracking
AWS Key Management

There is no easy way to see what all is running and what the ecosystem connections are
AWS Config

Additional tracking on top of AWS CloudTrail
AWS Service Catalog

CI and CD
AWS CodeCommit
AWS CodePipeline
AWS CodeDeploy

AWS Aurora  (Cloud native DB)


20141113: AWS Lambda Compute Service
After you ask to be in the preview
https://console.aws.amazon.com/lambda/home?region=us-east-1#/create

get up to speed on node.js
https://aws.amazon.com/sdk-for-node-js/



Monday, August 4, 2014

Multitenancy, JAX-RS 2.0, JPA 2.1, JSON-P 1.0, WebSocket and Cassandra on WebLogic 12.1.3

Oracle WebLogic 12.1.3 shipped on 23 June 2014.
This release is significant because it has partial Java EE 7 support ( JAX-RS 2.0,WebSocket (WS protocol) support, JSON-P and JPA 2.1 spec support). As is normal oracle procedure the first release into a partial EE upgrade requires shared library enablement and movement of the JPA SPI injection jar at the top of the server class path - so CMP persistence will work with EclipseLink 2.5 and Hibernate 4.3.
We can now inject 2.1 version of Hibernate 4.3 and EclipseLink 2.5 entityManagers.  EclipseLink 2.5  will allow us to use multi-tenancy and JPA-RS 2.0 annotations.

See the Oracle WebLogic 12.1.3 whitepaper
http://www.oracle.com/technetwork/middleware/weblogic/overview/weblogic-server-12-1-3-whitepaper-2227074.pdf

Up your heap and permgen in
user_projects/domains/base_domain/bin/setDomainEnv.sh
-JAVA Memory arguments: -Xms256m -Xmx512m -XX:CompileThreshold=8000 -XX:PermSize=128m  -XX:MaxPermSize=256m
+JAVA Memory arguments: -Xms256m -Xmx1024m -XX:CompileThreshold=8000 -XX:PermSize=512m  -XX:MaxPermSize=512m

JAX-RS 2.0

Enable JAX-RS 2.0 by deploying the 2.0 library war on 12.1.3
Library jax-rs(2.0,2.5.1)ActiveLibraryAdminServer100
You will see
<20-Aug-2014 5:52:17 o'clock PM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RUNNING.>
Aug 20, 2014 5:52:35 PM org.glassfish.jersey.server.ApplicationHandler initialize
INFO: Initiating Jersey application, version Jersey: 2.5.1 2014-01-02 13:43:00...

Add the following to your web.xml
<pre>
  <servlet>
    <description>JAX-RS Tools Generated - Do not modify</description>
    <servlet-name>JAX-RS Servlet</servlet-name>
    <!-- servlet-class>javax.ws.rs.core.Application</servlet-class-->
    <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>JAX-RS Servlet</servlet-name>
    <url-pattern>/jaxrs/*</url-pattern>
    <url-pattern>/rest/*</url-pattern>
  </servlet-mapping>
</pre>
Add the following to your weblogic.xml
<pre>
   <wls:library-ref>
      <wls:library-name>jax-rs</wls:library-name>
      <wls:specification-version>2.0</wls:specification-version>
   </wls:library-ref>
</pre>
Use wlserver\orasocket\scripts\orasocket.min.js

JPA 2.1



JSON-P 1.0



WebSocket Protocol



Cassandra 2.0 NoSQL column DB
https://code.google.com/a/apache-extras.org/p/cassandra-jdbc/downloads/list
http://stackoverflow.com/questions/19042825/is-cql-jdbc-1-2-5-compatible-with-cassandra-thrift-2-0-1-and-cassandra-clientuti
http://stackoverflow.com/questions/15983190/cassandra-client-java-apis
http://code.google.com/p/kundera/
https://github.com/impetus-opensource/Kundera
https://hibernate.atlassian.net/browse/OGM-122
[OGM-122] Support Cassandra as Datastore - Hibernate JIRA



SQL

select count(1) from gps_record where coalesce(geohash, '') <> ''





Tuesday, July 22, 2014

LDD: Living the Developer Dream

In order to live the developer dream - you need to do a lot of things I won't discuss here.  However the following couple suggestions may help you get there.

Start using the cloud - but until you get there, setup your own server on site.

Java Remote Diagnostics

Setting up the hosting server
- add the following java properties to your JVM - preferably on the WebLogic server "server start | arguments" tab

-Dcom.sun.management.jmxremote.port=13333 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false

- see the ports in your jvm process
obrienlabs-mbp15:security michaelobrien$ ps -ef | grep java

  501 79388 79343   0 11:57am ??         0:12.75 /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/bin/java -client -Xms256m -Xmx512m -XX:MaxPermSize=256m -Dweblogic.Name=AdminServer -Djava.security.policy=/Users/michaelobrien/Oracle/Middleware/Oracle_HomeC/wlserver/server/lib/weblogic.policy -Xverify:none -Djava.endorsed.dirs=/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/endorsed:/Users/michaelobrien/Oracle/Middleware/Oracle_HomeC/oracle_common/modules/endorsed -da -Dwls.home=/Users/michaelobrien/Oracle/Middleware/Oracle_HomeC/wlserver/server -Dweblogic.home=/Users/michaelobrien/Oracle/Middleware/Oracle_HomeC/wlserver/server -Dcom.sun.management.jmxremote.port=13333 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false weblogic.Server

Setting up the diagnostic client

obrienlabs-mbp15:bin michaelobrien$ jvisualvm -J-Dnetbeans.system_socks_proxy=localhost:13333 -J-Djava.net.useSystemProxies=true


You should see the following from a VMWare Redhat client connected to the host OSX 10.9 server running the JVM.

Java Remote Debugging


http://en.wikipedia.org/wiki/NSA_ANT_catalog


Wednesday, July 2, 2014

Hexoskin Heart Rate, ECG,VO2 and activity capture

Hexoskin is an excellent product from Carré technologies that I find extremely useful.

Pros
- real time iOS and Android monitoring 


Cons
- ECG data available online only by disconnecting/syncing in OSX/win7









Tuesday, June 17, 2014

APIs Frameworks and Tools

Alphabetical

JRuby

Play (Google) / SBT (Scala Build Tool - Typesafe)

C:\_dev\_play>play new test2
       _
 _ __ | | __ _ _  _
| '_ \| |/ _' | || |
|  __/|_|\____|\__ /
|_|            |__/

play 2.2.3 built with Scala 2.10.3 (running Java 1.7.0_51), http://www.playframework.com

The new application will be created in C:\_dev\_play\test2

What is the application name? [test2]
> redirect

Which template do you want to use for this new application?

  1             - Create a simple Scala application
  2             - Create a simple Java application

> 1
OK, application redirect is created.

Have fun!

C:\_dev\_play>cd test2

C:\_dev\_play\test2>dir
 Volume in drive C is OS
 Volume Serial Number is 4E12-1E48

 Directory of C:\_dev\_play\test2

29/05/2014  16:26    <DIR>          .
29/05/2014  16:26    <DIR>          ..
29/05/2014  16:25               141 .gitignore
24/07/2014  16:28    <DIR>          app
24/07/2014  16:28               140 build.sbt
24/07/2014  16:28    <DIR>          conf
24/07/2014  16:28    <DIR>          project
24/07/2014  16:28    <DIR>          public
29/05/2014  16:25               147 README
24/07/2014  16:28    <DIR>          test
               3 File(s)            428 bytes
               7 Dir(s)   3,725,914,112 bytes free

C:\_dev\_play\test2>play
Getting org.scala-sbt sbt 0.13.0 ...
:: retrieving :: org.scala-sbt#boot-app
        confs: [default]
        43 artifacts copied, 0 already retrieved (12440kB/185ms)
[info] Loading project definition from C:\_dev\_play\test2\project
[info] Set current project to redirect (in build file:/C:/_dev/_play/test2/)
       _
 _ __ | | __ _ _  _
| '_ \| |/ _' | || |
|  __/|_|\____|\__ /
|_|            |__/

play 2.2.3 built with Scala 2.10.3 (running Java 1.7.0_51), http://www.playframework.com

> Type "help play" or "license" for more information.
> Type "exit" or use Ctrl+D to leave this console.

[redirect] $ help play
Welcome to Play 2.2.3!

These commands are available:
-----------------------------
classpath                  Display the project classpath.
clean                      Clean all generated files.
compile                    Compile the current application.
console                    Launch the interactive Scala console (use :quit to exit).
dependencies               Display the dependencies summary.
dist                       Construct standalone application package.
exit                       Exit the console.
h2-browser                 Launch the H2 Web browser.
license                    Display licensing informations.
package                    Package your application as a JAR.
play-version               Display the Play version.
publish                    Publish your application in a remote repository.
publish-local              Publish your application in the local repository.
reload                     Reload the current application build file.
run <port>                 Run the current application in DEV mode.
test                       Run Junit tests and/or Specs from the command line
eclipse                    generate eclipse project file
idea                       generate Intellij IDEA project file
sh <command to run>        execute a shell command
start <port>               Start the current application in another JVM in PROD mode.
update                     Update application dependencies.

Type `help` to get the standard sbt help.

[redirect] $ publish
[info] Packaging C:\_dev\_play\test2\target\scala-2.10\redirect_2.10-1.0-SNAPSHOT-sources.jar ...
[info] Updating {file:/C:/_dev/_play/test2/}test2...
[info] Done packaging.
[info] Wrote C:\_dev\_play\test2\target\scala-2.10\redirect_2.10-1.0-SNAPSHOT.pom
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
  [info] Resolving com.typesafe.play#play-jdbc_2.10;2.2.3 ...
  [info] Resolving com.typesafe.play#play_2.10;2.2.3 ...
  [info] Resolving com.typesafe.play#sbt-link;2.2.3 ...
  [info] Resolving org.javassist#javassist;3.18.0-GA ...
  [info] Resolving com.typesafe.play#play-exceptions;2.2.3 ...
  [info] Resolving com.typesafe.play#templates_2.10;2.2.3 ...
  [info] Resolving com.github.scala-incubator.io#scala-io-file_2.10;0.4.2 ...
  [info] Resolving com.github.scala-incubator.io#scala-io-core_2.10;0.4.2 ...
  [info] Resolving com.jsuereth#scala-arm_2.10;1.3 ...
  [info] Resolving com.typesafe.play#play-iteratees_2.10;2.2.3 ...
  [info] Resolving org.scala-stm#scala-stm_2.10;0.7 ...
  [info] Resolving com.typesafe#config;1.0.2 ...
  [info] Resolving com.typesafe.play#play-json_2.10;2.2.3 ...
  [info] Resolving com.typesafe.play#play-functional_2.10;2.2.3 ...
  [info] Resolving com.typesafe.play#play-datacommons_2.10;2.2.3 ...
  [info] Resolving joda-time#joda-time;2.2 ...
  [info] Resolving org.joda#joda-convert;1.3.1 ...
  [info] Resolving com.fasterxml.jackson.core#jackson-annotations;2.2.2 ...
  [info] Resolving com.fasterxml.jackson.core#jackson-core;2.2.2 ...
  [info] Resolving com.fasterxml.jackson.core#jackson-databind;2.2.2 ...
  [info] Resolving org.scala-lang#scala-reflect;2.10.3 ...
  [info] Resolving io.netty#netty;3.7.1.Final ...
  [info] Resolving com.typesafe.netty#netty-http-pipelining;1.1.2 ...
  [info] Resolving org.slf4j#slf4j-api;1.7.5 ...
  [info] Resolving org.slf4j#jul-to-slf4j;1.7.5 ...
  [info] Resolving org.slf4j#jcl-over-slf4j;1.7.5 ...
  [info] Resolving ch.qos.logback#logback-core;1.0.13 ...
  [info] Resolving ch.qos.logback#logback-classic;1.0.13 ...
  [info] Resolving com.typesafe.akka#akka-actor_2.10;2.2.0 ...
  [info] Resolving com.typesafe.akka#akka-slf4j_2.10;2.2.0 ...
  [info] Resolving org.apache.commons#commons-lang3;3.1 ...
  [info] Resolving com.ning#async-http-client;1.7.18 ...
  [info] Resolving oauth.signpost#signpost-core;1.2.1.2 ...
  [info] Resolving commons-codec#commons-codec;1.3 ...
  [info] Resolving oauth.signpost#signpost-commonshttp4;1.2.1.2 ...
  [info] Resolving org.apache.httpcomponents#httpcore;4.0.1 ...
  [info] Resolving org.apache.httpcomponents#httpclient;4.0.1 ...
  [info] Resolving commons-logging#commons-logging;1.1.1 ...
  [info] Resolving xerces#xercesImpl;2.11.0 ...
  [info] Resolving xml-apis#xml-apis;1.4.01 ...
  [info] Resolving javax.transaction#jta;1.1 ...
  [info] Resolving com.jolbox#bonecp;0.8.0.RELEASE ...
  [info] Resolving com.google.guava#guava;14.0.1 ...
  [info] Resolving com.h2database#h2;1.3.172 ...
  [info] Resolving tyrex#tyrex;1.0.1 ...
  [info] Resolving com.typesafe.play#anorm_2.10;2.2.3 ...
  [info] Resolving com.typesafe.play#play-cache_2.10;2.2.3 ...
  [info] Resolving net.sf.ehcache#ehcache-core;2.6.6 ...
  [info] Resolving com.typesafe.play#play-test_2.10;2.2.3 ...
  [info] Resolving junit#junit;4.11 ...
  [info] Resolving org.hamcrest#hamcrest-core;1.3 ...
  [info] Resolving org.specs2#specs2_2.10;2.1.1 ...
  [info] Resolving org.scalaz#scalaz-core_2.10;7.0.2 ...
  [info] Resolving org.scalaz#scalaz-concurrent_2.10;7.0.2 ...
  [info] Resolving org.scalaz#scalaz-effect_2.10;7.0.2 ...
  [info] Resolving com.novocode#junit-interface;0.10 ...
  [info] Resolving org.scala-tools.testing#test-interface;0.5 ...
  [info] Resolving com.google.code.findbugs#jsr305;2.0.1 ...
  [info] Resolving org.fluentlenium#fluentlenium-festassert;0.8.0 ...
  [info] Resolving org.fluentlenium#fluentlenium-core;0.8.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-java;2.32.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-android-driver;2.32.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-remote-driver;2.32.0 ...
  [info] Resolving cglib#cglib-nodep;2.1_3 ...
  [info] Resolving org.json#json;20080701 ...
  [info] Resolving org.seleniumhq.selenium#selenium-api;2.32.0 ...
  [info] Resolving org.apache.httpcomponents#httpclient;4.2.1 ...
  [info] Resolving org.apache.httpcomponents#httpcore;4.2.1 ...
  [info] Resolving commons-codec#commons-codec;1.6 ...
  [info] Resolving org.apache.commons#commons-exec;1.1 ...
  [info] Resolving net.java.dev.jna#jna;3.4.0 ...
  [info] Resolving net.java.dev.jna#platform;3.4.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-chrome-driver;2.32.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-htmlunit-driver;2.32.0 ...
  [info] Resolving net.sourceforge.htmlunit#htmlunit;2.12 ...
  [info] Resolving xalan#xalan;2.7.1 ...
  [info] Resolving xalan#serializer;2.7.1 ...
  [info] Resolving commons-collections#commons-collections;3.2.1 ...
  [info] Resolving org.apache.httpcomponents#httpmime;4.2.3 ...
  [info] Resolving net.sourceforge.htmlunit#htmlunit-core-js;2.12 ...
  [info] Resolving net.sourceforge.nekohtml#nekohtml;1.9.18 ...
  [info] Resolving net.sourceforge.cssparser#cssparser;0.9.9 ...
  [info] Resolving org.w3c.css#sac;1.3 ...
  [info] Resolving commons-io#commons-io;2.2 ...
  [info] Resolving org.eclipse.jetty#jetty-websocket;8.1.9.v20130131 ...
  [info] Resolving org.eclipse.jetty#jetty-util;8.1.9.v20130131 ...
  [info] Resolving org.eclipse.jetty#jetty-io;8.1.9.v20130131 ...
  [info] Resolving org.eclipse.jetty#jetty-http;8.1.9.v20130131 ...
  [info] Resolving org.seleniumhq.selenium#selenium-firefox-driver;2.32.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-ie-driver;2.32.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-iphone-driver;2.32.0 ...
  [info] Resolving org.seleniumhq.selenium#selenium-safari-driver;2.32.0 ...
  [info] Resolving org.webbitserver#webbit;0.4.14 ...
  [info] Resolving org.seleniumhq.selenium#selenium-support;2.32.0 ...
  [info] Resolving org.easytesting#fest-assert;1.4 ...
  [info] Resolving org.easytesting#fest-util;1.1.6 ...
  [info] Resolving com.typesafe.play#play-docs_2.10;2.2.3 ...
  [info] Resolving com.typesafe.play#play-doc_2.10;1.0.3 ...
  [info] Resolving org.pegdown#pegdown;1.4.0 ...
  [info] Resolving org.parboiled#parboiled-java;1.1.5 ...
  [info] Resolving org.parboiled#parboiled-core;1.1.5 ...
  [info] Resolving org.ow2.asm#asm;4.1 ...
  [info] Resolving org.ow2.asm#asm-tree;4.1 ...
  [info] Resolving org.ow2.asm#asm-analysis;4.1 ...
  [info] Resolving org.ow2.asm#asm-util;4.1 ...
  [info] Resolving commons-io#commons-io;2.4 ...
  [info] Resolving org.scala-lang#scala-compiler;2.10.3 ...
  [info] Resolving org.scala-lang#jline;2.10.3 ...
  [info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] :: delivering :: redirect#redirect_2.10;1.0-SNAPSHOT :: 1.0-SNAPSHOT :: integration :: Thu Jul 24 16:29:08 EDT 2014
[info]  delivering ivy file to C:\_dev\_play\test2\target\scala-2.10\ivy-1.0-SNAPSHOT.xml
[info] Compiling 5 Scala sources and 1 Java source to C:\_dev\_play\test2\target\scala-2.10\classes...
[info] Main Scala API documentation to C:\_dev\_play\test2\target\scala-2.10\api...
[info] 'compiler-interface' not yet compiled for Scala 2.10.3. Compiling...
[info]   Compilation completed in 8.149 s
model contains 17 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging C:\_dev\_play\test2\target\scala-2.10\redirect_2.10-1.0-SNAPSHOT-javadoc.jar ...
[info] Done packaging.
[info] Packaging C:\_dev\_play\test2\target\scala-2.10\redirect_2.10-1.0-SNAPSHOT.jar ...
[info] Done packaging.
[trace] Stack trace suppressed: run last *:publishConfiguration for the full output.
[error] (*:publishConfiguration) Repository for publishing is not specified.
[error] Total time: 16 s, completed 24-Jul-2014 4:29:21 PM


[redirect] $ start 9001

(Starting server. Type Ctrl+D to exit logs, the server will remain in background)

Play server process ID is 8440
[info] play - Application started (Prod)
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9001


http://127.0.0.1:9001/
Your new application is ready.

Categorized

Javascript / CSS


Java EE 7


Java EE 6


Java EE 5


jBPM


Sunday, June 15, 2014

Upgrading to WebLogic 12c (12.1.2 or 12.1.3) from 10.3.5 or 10.3.6

    If you are upgrading from an older 11g 10.3.5 or 10.3.6 version of WebLogic Server to the latest 12.1.2 (EE6) or 12.1.3 (EE6 + partial EE7) version - the following references may be of assistance.

Setup

You can run 2 WebLogic domains simultaneously by running them on different ports.  Put one of the domains on admin port 7001 and the 2nd on admin port 17001, managed server ports 7101/7201 and nodemanager port 5556.


Create a (non-TX with properties: "Supports Global Transactions" global TX with one-phase commit) datasource to (for example postgreSQL) to be used by JMS called "MWJMSds".
Create a JDBC persistent store for each migratable server - but you will only be able to use datasources that are completely non-transactional.  Turn off the checkbox on the DS to see it in the dropdown of the persistent store create page.
The following table should be created "MW1WLStore" using a MW1 and MW2 prefix but we get.


one or more registered update listeners reported activation problems. The following failures occurred: -- The following failures occurred: -- javax.naming.NameNotFoundException: While trying to lookup 'jms.MWJDBCDerbyDS2' didn't find subcontext 'jms'. Resolved ''; remaining name 'jms/MWJDBCDerbyDS2' weblogic.management.DeploymentException: javax.naming.NameNotFoundException: While trying to lookup 'jms.MWJDBCDerbyDS2' didn't find subcontext 'jms'. Resolved ''; remaining name 'jms/MWJDBCDerbyDS2'

Make sure you target the managed server - not the admin server

JDBCStore
MW_ManagedServer_1 (migratable)

Create JMS servers and target the persistent store and managed server target


JDBCStore-0
MW_ManagedServer_1 (migratable)
MW_ManagedServer_1


Create a JMS Module
Create a subdeployment - target only the 2 JMS servers
Create a JMS distributed queue - use advanced targeting and select the subdeployment.
Go back into the JMS queue | SubDeployment and notice that the subdeployment is reset to "none" - this seems to be a new bug in WebLogic 12.1.2.0 - if you resave this page you will reset the subdeployment and target


Uniform Distributed Queue
jms/DistributedQueue-1
subdeployment
JMSServer-1, JMSServer-2



If I deselect one of the JMS servers in the subdeployment setup page - then the subdeployment reappears on the subdeployment dropdown of the distributed queue.
Other possible workarounds like changing the target of the subdeployment to the cluster (which inherits the JMS server(s)) does not work.

There are many features not available in default targeting - therefore we need to get multi-server subdeployment working.
http://docs.oracle.com/middleware/1212/wls/JMSAD/dynamic_messaging.htm#CHDHFDEI

weblogic_timers table:
Verify that the datasource is set on the cluster | configuration | scheduling page

See
https://community.oracle.com/thread/2502444

Architecture

The server consists of a cluster where each physical node has a nodemanager, 2 managed servers and 2 JMS servers.

Design Issues

DI 1: JMS Server failover via Migratable Target

DI 2: Time for all managed servers and DB must be synchronized


Verification Test Plan:

JMS

Uniform Distributed Queues:
The documentation for 12.1.2 states that default targeting should be deferred in favour of advanced targeting involving subdeployments.
http://docs.oracle.com/middleware/1212/wls/JMSAD/best_practice.htm#CACFGECE
However there is an issue where the subdeployment that targets more than one JMS server is not selectable as a target for UDQ's.

Targeting the cluster instead of a subdeployment is very limited in functionality - therefore we need to get multi-server subdeployment working.
http://docs.oracle.com/middleware/1212/wls/JMSAD/dynamic_messaging.htm#CHDHFDEI

Cluster | Configuration | Singleton Services

JPA 2.0

Although JPA 2.0 has been available since the 15 Jan 2010 release of WebLogic 10.3.4 - upgrading from JPA 1.0 may experience the following issues.

Issues:

Listen Address Unspecified:
The following warning means that multiple IP's are configured for the cluster
<Jun 15, 2014 8:10:48 PM EDT> <Warning> <Cluster> <BEA-003121> <Unicast cluster may not function correctly as the listen address of server MW_ManagedServer_1 is not specified.> 

References:

http://docs.oracle.com/middleware/1212/wls/NOTES/index.html
http://docs.oracle.com/middleware/1212/wls/JMSPG/index.html
http://middlewaresnippets.blogspot.ca/2013/11/jms-migration-in-weblogic-server-12c.html

Thursday, March 27, 2014

Apple MacBook Pro is the key to living the developer dream - have your iOS, your Android and Java EE

      My MacBook Pro is awesome.   On a single machine I am able to develop Java EE applications, write Android apps and of course evolve iOS mobile applications.   The high end machine is a fusion of what a developer needs.
1) Unix based OS
2) iOS development
3) Java EE development
4) Android development
5) Screen Sharing replaces RDP
6) Vmware Fusion 6 allows for running licensed versions of Windows 7 and an unlimited number of OSX 10.9 VM's
7) When installing iOS 8 beta on an iphone 5s or ipod touch 5th - you will need to cycle the device in member central as well as remove old provisioning profiles in XCode 6 | Devices - re-add them and then reconnect the device.  You will then be able to run new "Swift" coded apps from XCode 6

1) Make sure to turn off native mode for WebLogic 12c node manager on the mac
NativeVersionEnabled=false
http://cerebro.com.au/2013/02/25/nodemanager-on-mac-osx/
2) for Insufficient disk space error
use the flag -Dspace.detection=false
http://docs.oracle.com/cd/E21764_01/doc.1111/e14775/weblogic_server_issues.htm#CIHIHFCB

Friday, March 21, 2014

Using postgreSQL with WebLogic 12c and EclipseLink 2.4 or Hibernate 4.2

This article describes how to get the postgreSQL 9.3 database running on WebLogic 12.1.2 using EclipseLink 2.4 or Hibernate 4.2 as the JPA persistence provider

Tutorial assumes EclipseLink 2.4.2.v20130514-5956486 running on WebLogic 12.1.2.1 on OSX 10.9 using postgreSQL 9.3 the postgreSQL
JDBC driver jar does not come with WebLogic http://docs.oracle.com/cd/E15051_01/wls/docs103/jdbc_admin/third_party_drivers.html
modify the class http://docs.oracle.com/cd/E15051_01/wls/docs103/admin_ref/weblogicServer.html#ModifyingClasspaht
Download the latest postgreSQL 9.3 JDBC 4.1 jar from
http://jdbc.postgresql.org/download.htmlpostgresql-9.3-1101.jdbc41.jar
Install postgreSQL 9.3 on the mac
Create a database by running Applications/PostgreSQL 9.3/pgAdmin III

Create a datasource in the WebLogic console

JAVA SE RESOURCE_LOCAL persistence.xml
org.eclipse.persistence.jpa.PersistenceProvider

results
[EL Info]: 2014-03-19 18:03:22.718--ServerSession(1898536649)--Thread(Thread[main,5,main])--EclipseLink, version: Eclipse Persistence Services - 2.4.2.v20130514-5956486 [EL Config]: connection: 2014-03-19 18:03:22.722--ServerSession(1898536649)--Connection(223623898)--Thread(Thread[main,5,main])--connecting(DatabaseLogin( platform=>PostgreSQLPlatform user name=> "postgres" datasource URL=> "jdbc:postgresql://localhost:5432/obrienlabs"

JAVA EE JTA Container Managed persistence.xml
org.eclipse.persistence.jpa.PersistenceProvider

results
[EL Config]: connection: 2014-03-19 19:50:20.599--ServerSession(1214490893)--Connection(1145000203)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--connecting(DatabaseLogin( platform=>PostgreSQLPlatform user name=> "" connector=>JNDIConnector datasource name=>null )) [EL Config]: connection: 2014-03-19 19:50:20.599--ServerSession(1214490893)--Connection(417451054)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--Connected: jdbc:postgresql://127.0.0.1:5432/obrienlabs User: postgres Database: PostgreSQL Version: 9.3.3 Driver: PostgreSQL Native Driver Version: PostgreSQL 9.3 JDBC4.1 (build 1101) [EL Finest]: sequencing: 2014-03-19 19:50:20.599--ServerSession(1214490893)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--sequencing connected, state is Preallocation_Transaction_NoAccessor_State

Hibernate 4.2.12 on WebLogic 12.1.2.0

Hibernate 4.3 will not run as a container managed persistence context on WebLogic 12.1.2.0 because only the JPA 2.0 version of the EntityManager proxy is supported (12.1.3.0 should support JPA 2.1).  Therefore using Hibernate 4.2 is recommended for 12c.

persistence.xml












<property name="hibernate.transaction.jta.platform" value="org.hibernate.service.jta.platform.internal.WeblogicJtaPlatform"/>
<property name="hibernate.dialect" value="org.hibernate.dialect.PostgreSQLDialect"/>   


Running any JPA provider that supports the 2.0 spec

Notes:
1) For polymorphic inheritance - the workaround annotation @ForceDescriminator is definitely your friend


Friday, February 21, 2014

All In CPU Performance Benchmarks

Java on

Macbook Pro 15 (8 core 2.6 GHz i7-4960) 16G RAM | OS 7.0.4 | Java 1.7.0_51 64-bit

38166 ms for Collatz  split: 16
38740 ms for Collatz  split: 32
37234 ms for Collatz  split: 64
35005 ms for Collatz  split: 128
34437 ms for Collatz  split: 256
32271 ms for Collatz  split: 512
31803 ms for Collatz  split: 1024
31707 ms for Collatz  split: 2048
31085 ms for Collatz  split: 4096
31469 ms for Collatz  split: 8192
32078 ms for Collatz  split: 16384
33302 ms for Collatz  split: 32768
32765 ms for Collatz  split: 65536
31821 ms for Collatz  split: 131072
31513 ms for Collatz  split: 262144
32498 ms for Collatz  split: 524288
33560 ms for Collatz  split: 1048576
36520 ms for Collatz  split: 2097152
34296 ms for Collatz  split: 4194304
34482 ms for Collatz  split: 8388608
40003 ms for Collatz  split: 16777216

61681 ms for Collatz  split: 33554432

Asus


Total Pageviews

Followers