tag:blogger.com,1999:blog-38102532368120625482024-03-13T00:24:12.584-07:00JPA PersistenceRunning JPA based Enterprise Java EE applications on various Application Server containersMichael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.comBlogger58125tag:blogger.com,1999:blog-3810253236812062548.post-20884624991969309342019-03-25T10:13:00.001-07:002019-07-28T07:00:06.557-07:00FPGA development for parallel processing<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px}
</style>
<br />
<div class="p1">
see <a href="http://wiki.obrienlabs.cloud/display/DEV/FPGA">http://wiki.obrienlabs.cloud/display/DEV/FPGA</a><br />
Had a discussion with a trusted mentor of mine - spanning hardware and software home based work today - specifically about restarting FPGA work I looked at in 2010.<br />
<br />
Move to OPENCL for FPGA based</div>
<div class="p1">
<a href="https://www.xilinx.com/content/dam/xilinx/imgs/products/zynq/zynq-eg-block.PNG">https://www.xilinx.com/content/dam/xilinx/imgs/products/zynq/zynq-eg-block.PNG</a></div>
<div class="p1">
$249</div>
<div class="p1">
<a href="https://www.xilinx.com/products/boards-and-kits/1-vad4rl.html">https://www.xilinx.com/products/boards-and-kits/1-vad4rl.html</a></div>
<div class="p1">
voucher for 1 year of<span class="Apple-converted-space"> </span></div>
<div class="p1">
60d-eval/$1k</div>
<div class="p1">
<a href="https://www.xilinx.com/products/design-tools/software-zone/sdsoc.html">https://www.xilinx.com/products/design-tools/software-zone/sdsoc.html</a></div>
<div class="p1">
<a href="https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842491/Boards+and+Kits+Voucher+Licensing">https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842491/Boards+and+Kits+Voucher+Licensing</a></div>
<div class="p2">
or<br />
$700 PCI based<br />
<a href="https://www.xilinx.com/products/boards-and-kits/ek-s6-sp605-g.html">https://www.xilinx.com/products/boards-and-kits/ek-s6-sp605-g.html</a><br />
or<br />
$40 Lattice based 7k lookup table<br />
<a href="https://www.sparkfun.com/products/14829">https://www.sparkfun.com/products/14829</a></div>
<div class="p2">
<br /></div>
<div class="p1">
Deprecate all non-FPGA work below</div>
<div class="p1">
<a href="http://javahardware.blogspot.com/">http://javahardware.blogspot.com/</a></div>
<div class="p1">
<a href="http://kilocore.blogspot.com/">http://kilocore.blogspot.com/</a></div>
<div class="p1">
<a href="http://obrienlabs.blogspot.com/2010/12/altera-cyclone-iii-development-kits.html">http://obrienlabs.blogspot.com/2010/12/altera-cyclone-iii-development-kits.html</a></div>
<div class="p1">
<a href="https://eclipsejpa.blogspot.com/2012/11/java-7-se-parallel-processing-cluster.html">https://eclipsejpa.blogspot.com/2012/11/java-7-se-parallel-processing-cluster.html</a></div>
<br />Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-49387050379471342202019-03-10T21:07:00.000-07:002019-10-05T06:17:28.020-07:00Java 8 Lambda Parallel Streams :: Use thread safe collectionssee <a href="http://wiki.obrienlabs.cloud/display/DEV/Design+Issues+-+Case+Studies#DesignIssues-CaseStudies-Java8LambdaParallelStreams::Usethreadsafecollections">http://wiki.obrienlabs.cloud/display/DEV/Design+Issues+-+Case+Studies#DesignIssues-CaseStudies-Java8LambdaParallelStreams::Usethreadsafecollections</a><br />
<br />
Changing code to use parallelStream from stream is not as simple as it looks - there are several design principles that need to be accounted for.<br />
<br />
The parallel stream code in the lambda libraries wrap the ForkJoin framework from Java 7. There are several issues that will affect performance and accuracy of your code.<br />
<br />
Using non-threadsafe collections - this will show up as randomly varying results and modifications of Lists and Maps if they use out of the box non-threadsafe implementations like ArrayList and HashMap. Stick to CopyOnWriteArrayList and ConcurrentHashMap.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjil7gAT1M0wAqj7Tnyuedjxs_CV4nPcXByP1rDmXOAFkJ42D_Ea4IpmsMu1niwaxPrWoXZXW8KtV6KnRgoiRrpJPrfeXdwOwLDFfAQMaid1YoZ0nGmDAE4Q787NHk-zHjm4pVv7SmtR-bZ/s1600/Screenshot+2019-03-11+00.17.48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1083" data-original-width="1600" height="432" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjil7gAT1M0wAqj7Tnyuedjxs_CV4nPcXByP1rDmXOAFkJ42D_Ea4IpmsMu1niwaxPrWoXZXW8KtV6KnRgoiRrpJPrfeXdwOwLDFfAQMaid1YoZ0nGmDAE4Q787NHk-zHjm4pVv7SmtR-bZ/s640/Screenshot+2019-03-11+00.17.48.png" width="640" /></a></div>
<br />
<br />
For example<br />
<br />
<br />
Using a small dataset for parallel operations may be slower than sequential code because of the parallel framework overhead. Try to use data sets that are at lease 10k in size<br />
<br />
For example - filtering on 10 Million biometric records.<br />
<br />
<br />
<br />Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-71507916998167899272019-03-08T12:29:00.000-08:002020-04-01T17:06:22.112-07:00Testing for Radon 222 radioactive gas levels above 100 Bq/m3 in the home as a result of the U235-Ra226-Rn222 decay chain<a href="http://wiki.obrienlabs.cloud/display/DEV/Science#Science-Radon">http://wiki.obrienlabs.cloud/display/DEV/Science#Science-Radon</a><br />
<br />
<div style="background-color: white; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: small;">
I have been meaning to test the radon(222) levels for a while - it looks like we have elevated gas levels in the lower airspace in the basement. I just started preliminary testing before placing 6+ month laboratory grade sensors. The results from the Airthings sensor even after 3 days is interesting. The plan is to move the meter around daily to get a 3d view of the house before winter ends.<br />
<br />
Have an ERV running.<br />
<br />
The issue is unventilated confined lower spaces - we are looking for a number lower than 100 Bq/m(3) - but any level is bad. A Bq is defined as 1 alpha particle (helium nucleus) per second per cubic meter.</div>
<div style="background-color: white; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: small;">
<br />
<b>Results:</b><br />
<b><br /></b>
20190307:0823 - bedroom 1st floor 1m above floor (12h)<br />
79<br />
20190307:2240 - basement by work area - 50cm above floor<br />
67<br />
20190308:0858 - basement back by furnace by crack in floor - right on the floor<br />
666<br />
20190308:1243 - basement back by furnace by crack in floor - on 30cm box<br />
732<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOAXEV4kFG4QJTieX_dK8yP96NB5UiLG-e8FvSEvbeMXKgaVj4wZKBneuXjJtFEizWGfHYoZFnu86A20JlPWhBqy07oJGtGBGSBedVa3YtPblKrgrSzOqaKAZYvoEpOl5fxvGADz1y9JI7/s1600/IMG_20190308_0858249.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="853" data-original-width="1279" height="212" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOAXEV4kFG4QJTieX_dK8yP96NB5UiLG-e8FvSEvbeMXKgaVj4wZKBneuXjJtFEizWGfHYoZFnu86A20JlPWhBqy07oJGtGBGSBedVa3YtPblKrgrSzOqaKAZYvoEpOl5fxvGADz1y9JI7/s320/IMG_20190308_0858249.jpg" width="320" /></a></div>
<br />
<br />
<br /></div>
<div dir="ltr" style="background-color: white; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: small;">
highest link<br />
<div>
<a data-saferedirecturl="https://www.google.com/url?q=https://healthfinder.gov/HealthTopics/Category/pregnancy/getting-ready-for-your-baby/test-your-home-for-radon-quick-tips&source=gmail&ust=1552163184214000&usg=AFQjCNFmKRmhRvmVikI0KR18bePpf2_f3Q" href="https://healthfinder.gov/HealthTopics/Category/pregnancy/getting-ready-for-your-baby/test-your-home-for-radon-quick-tips" style="color: #1155cc;" target="_blank">https://healthfinder.gov/<wbr></wbr>HealthTopics/Category/<wbr></wbr>pregnancy/getting-ready-for-<wbr></wbr>your-baby/test-your-home-for-<wbr></wbr>radon-quick-tips</a></div>
<div>
<br /></div>
<div>
Canada specific</div>
<div>
<br /></div>
<div>
<a data-saferedirecturl="https://www.google.com/url?q=https://www.canada.ca/en/health-canada/services/environmental-workplace-health/reports-publications/radiation/radon-your-home-health-canada-2009.html&source=gmail&ust=1552163184214000&usg=AFQjCNGUaKslrntIjiZdo6b36GabraCSpA" href="https://www.canada.ca/en/health-canada/services/environmental-workplace-health/reports-publications/radiation/radon-your-home-health-canada-2009.html" style="color: #1155cc;" target="_blank">https://www.canada.ca/en/<wbr></wbr>health-canada/services/<wbr></wbr>environmental-workplace-<wbr></wbr>health/reports-publications/<wbr></wbr>radiation/radon-your-home-<wbr></wbr>health-canada-2009.html</a></div>
<div>
we need to be below 200 Bq/m(3) but even 50 is still dangerous</div>
<div>
<br /></div>
<div>
getting these (2) from Canada - to send in for a 90+ day check</div>
<div>
<a data-saferedirecturl="https://www.google.com/url?q=https://www.radondetect.ca/product.php?productid%3D2&source=gmail&ust=1552163184214000&usg=AFQjCNGybrNry4ynnPxByOxrrpl3UEeh1g" href="https://www.radondetect.ca/product.php?productid=2" style="color: #1155cc;" target="_blank">https://www.radondetect.ca/<wbr></wbr>product.php?productid=2</a></div>
<div>
and likely this one for continuous monitoring</div>
<div>
<div>
<a data-saferedirecturl="https://www.google.com/url?q=https://www.amazon.ca/Corentium-Airthings-Detector-Canadian-Version/dp/B00H30TLPA/ref%3Dsr_1_3?keywords%3Dairthings%2Bwave%26qid%3D1551632344%26s%3Dgateway%26sr%3D8-3&source=gmail&ust=1552163184219000&usg=AFQjCNEFjf1Bf5cmpjBILaQaao-Vi4GSug" href="https://www.amazon.ca/Corentium-Airthings-Detector-Canadian-Version/dp/B00H30TLPA/ref=sr_1_3?keywords=airthings+wave&qid=1551632344&s=gateway&sr=8-3" style="color: #1155cc;" target="_blank">https://www.amazon.ca/<wbr></wbr>Corentium-Airthings-Detector-<wbr></wbr>Canadian-Version/dp/<wbr></wbr>B00H30TLPA/ref=sr_1_3?<wbr></wbr>keywords=airthings+wave&qid=<wbr></wbr>1551632344&s=gateway&sr=8-3</a></div>
<div>
2 of</div>
<div>
<a data-saferedirecturl="https://www.google.com/url?q=https://www.amazon.ca/AccuStarCanada-1AT-Alpha-Long-Term-Canadians/dp/B00FBLWZJC/ref%3Dpd_bxgy_328_img_2/139-3705472-5145464?_encoding%3DUTF8%26pd_rd_i%3DB00FBLWZJC%26pd_rd_r%3Da8e79c23-3dd5-11e9-a69f-f727b54b72aa%26pd_rd_w%3DNw5CK%26pd_rd_wg%3D8ZZ1k%26pf_rd_p%3D163ac593-0f51-4882-9080-de6873217880%26pf_rd_r%3DJDVEJEQTKCC3TNXMYN7S%26psc%3D1%26refRID%3DJDVEJEQTKCC3TNXMYN7S&source=gmail&ust=1552163184219000&usg=AFQjCNEDpLBLmlS7nI8YPvZvxTu1-K744w" href="https://www.amazon.ca/AccuStarCanada-1AT-Alpha-Long-Term-Canadians/dp/B00FBLWZJC/ref=pd_bxgy_328_img_2/139-3705472-5145464?_encoding=UTF8&pd_rd_i=B00FBLWZJC&pd_rd_r=a8e79c23-3dd5-11e9-a69f-f727b54b72aa&pd_rd_w=Nw5CK&pd_rd_wg=8ZZ1k&pf_rd_p=163ac593-0f51-4882-9080-de6873217880&pf_rd_r=JDVEJEQTKCC3TNXMYN7S&psc=1&refRID=JDVEJEQTKCC3TNXMYN7S" style="color: #1155cc;" target="_blank">https://www.amazon.ca/<wbr></wbr>AccuStarCanada-1AT-Alpha-Long-<wbr></wbr>Term-Canadians/dp/B00FBLWZJC/<wbr></wbr>ref=pd_bxgy_328_img_2/139-<wbr></wbr>3705472-5145464?_encoding=<wbr></wbr>UTF8&pd_rd_i=B00FBLWZJC&pd_rd_<wbr></wbr>r=a8e79c23-3dd5-11e9-a69f-<wbr></wbr>f727b54b72aa&pd_rd_w=Nw5CK&pd_<wbr></wbr>rd_wg=8ZZ1k&pf_rd_p=163ac593-<wbr></wbr>0f51-4882-9080-de6873217880&<wbr></wbr>pf_rd_r=JDVEJEQTKCC3TNXMYN7S&<wbr></wbr>psc=1&refRID=<wbr></wbr>JDVEJEQTKCC3TNXMYN7S</a></div>
<br />
<div>
</div>
<br />
<div style="-webkit-text-stroke-width: 0px; background-color: white; color: #222222; font-family: arial, helvetica, sans-serif; font-size: small; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
<div style="margin: 0px;">
<br /></div>
<div>
and optionally the smartphone only one - with time</div>
<div>
<a data-saferedirecturl="https://www.google.com/url?q=https://www.amazon.ca/Airthings-AS-2900-Smart-Detector/dp/B01MSEQYY9/ref%3Dsr_1_1?keywords%3Dairthings%2Bwave%26qid%3D1551632344%26s%3Dgateway%26sr%3D8-1&source=gmail&ust=1552163184219000&usg=AFQjCNFYuX0YZtxakB55IVYDLoUngTtAng" href="https://www.amazon.ca/Airthings-AS-2900-Smart-Detector/dp/B01MSEQYY9/ref=sr_1_1?keywords=airthings+wave&qid=1551632344&s=gateway&sr=8-1" style="color: #1155cc;" target="_blank">https://www.amazon.ca/<wbr></wbr>Airthings-AS-2900-Smart-<wbr></wbr>Detector/dp/B01MSEQYY9/ref=sr_<wbr></wbr>1_1?keywords=airthings+wave&<wbr></wbr>qid=1551632344&s=gateway&sr=8-<wbr></wbr>1</a></div>
</div>
</div>
<div>
<br /></div>
<div>
<b>causes:</b></div>
<div>
Since we life on bedrock - we have a couple sources.</div>
<div>
The cracks in the basement are over crushed stone</div>
<div>
The water around the basement</div>
<div>
The soil around the house - origin - and the rocks like the stepping stones.</div>
<div>
Ventilation is critical - we should be good with floor vents on the 1st floor - it is the basement ventilation that may be an issue - our unfinished basement.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div>
<b>some 5 min science.</b></div>
<div>
Because I always hear about how old the canadian shield is here - good for earthquake stability and no shifting of the ground - but bad for having Uranium and Gold - </div>
<div>
we are on 4.5 billion year old bedrock</div>
<div>
<a data-saferedirecturl="https://www.google.com/url?q=http://web.nmsu.edu/~dshearer/geology/09_Geotime/Geotime-interact/GeoTimeScale/geotime-scale.htm&source=gmail&ust=1552163184214000&usg=AFQjCNHZw7zX6JO4EcNqEH0XIrUEswt7_Q" href="http://web.nmsu.edu/~dshearer/geology/09_Geotime/Geotime-interact/GeoTimeScale/geotime-scale.htm" style="color: #1155cc;" target="_blank">http://web.nmsu.edu/~dshearer/<wbr></wbr>geology/09_Geotime/Geotime-<wbr></wbr>interact/GeoTimeScale/geotime-<wbr></wbr>scale.htm</a> </div>
<div>
map of ottawa - see purple, blue</div>
<div>
<a data-saferedirecturl="https://www.google.com/url?q=http://hoopermuseum.earthsci.carleton.ca/constantinou/BED.HTM&source=gmail&ust=1552163184214000&usg=AFQjCNGwLmJJHkvjSOJIZukT0o7XHDGkyQ" href="http://hoopermuseum.earthsci.carleton.ca/constantinou/BED.HTM" style="color: #1155cc;" target="_blank">http://hoopermuseum.earthsci.<wbr></wbr>carleton.ca/constantinou/BED.<wbr></wbr>HTM</a></div>
<div>
<br /></div>
<div>
Radon comes from Uranium - which is mostly made in merging neutron stars when parts of the neutron star pieces up to 1km wide (essentially a large atomic nucleus) are released from the main 10km wide star. These pieces because they are under a lot less gravitational force form normal atomic matter - they condense into less dense isotopes like U235 and even gold/silver. A smaller portion of elements higher than iron are made during the bounce off the core of the collapse of a supernova in the last minute of a large > 20 solar mass start stops burning silicon. </div>
<div>
<br /></div>
<div>
Radon <a data-saferedirecturl="https://www.google.com/url?q=https://www.radon.com/radon_facts/&source=gmail&ust=1552163184214000&usg=AFQjCNHHt4DCrJZmw8oD7gLm-EQFE23Okw" href="https://www.radon.com/radon_facts/" style="color: #1155cc;" target="_blank">https://www.radon.com/radon_<wbr></wbr>facts/</a> turns into lead slowly - <a data-saferedirecturl="https://www.google.com/url?q=https://www.researchgate.net/figure/The-Basic-Radon-222-Rn-Decay-Chain-The-isotopes-and-their-atomic-masses-are-shown_fig1_51026112&source=gmail&ust=1552163184214000&usg=AFQjCNGihg3Z6_tmzpunf9rPrE9eW5_ZWQ" href="https://www.researchgate.net/figure/The-Basic-Radon-222-Rn-Decay-Chain-The-isotopes-and-their-atomic-masses-are-shown_fig1_51026112" style="color: #1155cc;" target="_blank">https://www.researchgate.net/<wbr></wbr>figure/The-Basic-Radon-222-Rn-<wbr></wbr>Decay-Chain-The-isotopes-and-<wbr></wbr>their-atomic-masses-are-shown_<wbr></wbr>fig1_51026112</a> is 9 times heavier than air and accumulates at the bottom of anything - it comes not necessarily out of the basement cracks - more from the sides of the basement from the dirt</div>
</div>
<div>
Cause of radium in the soil</div>
<div>
Radon (originally from decayed Uranium) is released during the recoil of an alpha particle decay (essentially a helium nucleus) from Radium 226 - the alpha particle itself is destructive radiation for cells - but it is the ejected radon atom as it moves from radium 226 through radon 222 (ejecting a helium atom) that causes issues for us - until radon 222 decays with a half life of 4 days into Polonium (very toxic). Polonium decays within 3 min into ordinary lead. Even though Radon 222 has a small half life of 4 days - it is under constant replacement because it is a gas and being ejected at a rate of 10-50 percent from the pores inside soil when the ejected radon is near the surface and the vector of the alpha particle emission is in the right angle - <a data-saferedirecturl="https://www.google.com/url?q=https://certmapper.cr.usgs.gov/data/PubArchives/radon/georadon/3.html&source=gmail&ust=1552163184214000&usg=AFQjCNGRPQr72Iwv6aIQA_TbL4RtFks-vw" href="https://certmapper.cr.usgs.gov/data/PubArchives/radon/georadon/3.html" style="color: #1155cc;" target="_blank">https://certmapper.cr.usgs.<wbr></wbr>gov/data/PubArchives/radon/<wbr></wbr>georadon/3.html</a></div>
<div>
<br /></div>
<div>
<div>
<br /></div>
<div>
<br /></div>
<div>
</div>
</div>
</div>
<br />
Links<br />
<a href="https://davidsuzuki.org/wp-content/uploads/2017/09/revisiting-canada-radon-guideline.pdf">https://davidsuzuki.org/wp-content/uploads/2017/09/revisiting-canada-radon-guideline.pdf </a>Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-64467641568704411192019-01-25T17:29:00.001-08:002019-10-05T06:17:36.840-07:00Experiment 1901 Sending Packets around the Earth see <a href="http://wiki.obrienlabs.cloud/display/DEV/Experiment+1901%3A+Sending+Packets+around+the+Earth">http://wiki.obrienlabs.cloud/display/DEV/Experiment+1901%3A+Sending+Packets+around+the+Earth</a><br />
<br />
I know this sounds weird but not to someone who lives science as a normal part each day. I would like to make sure my REST calls propagate around the globe - not just to a server on the other side of the earth and back again like from us-east-1 to the Singapore region but through us-west-2 through singapore - to europe and back to us-east-1.<br />
Sort of when I fly - in order to maximize the number of takeoffs and landings - I try to get the most number of stops - a direct flight is not the best way to get somewhere when testing latency - as in packets on the cable from Oregon to China are better routed through Hawaii.<br />
I liked Andrew Blum's "Tubes" book - which include details on the endpoints for sub-marine cables - <a href="https://www.amazon.com/Tubes-Journey-Internet-Andrew-Blum/dp/0061994952">https://www.amazon.com/Tubes-Journey-Internet-Andrew-Blum/dp/0061994952 </a> - I also ran into a bit of this propagation delay working at Nokia.<br />
<br />
First principles will be install proxies at several regions in Amazon EC2 - <a href="https://aws.amazon.com/about-aws/global-infrastructure/">https://aws.amazon.com/about-aws/global-infrastructure/</a><br />
<br />
see <a href="https://www.submarinecablemap.com/">https://www.submarinecablemap.com/</a><br />
This is the path we will take - hopefully no backtracking and ideally it is roughly valid - as we do not actually cross the equator - I will also try through Sau Paulo.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjbte3NwFqWOtW5CGd2pLfIxVF2ERWELeOSP8oesqeXAbqcrhVKyZXYjSeb4w9-_1ipCE4gnGS2n1qWLLLeN6o7y3YMJg2trfmq4goQOxbAaD5ofB5Y8R-XGPWFxwwdGlIZ8Cq-2Df1ucI/s1600/20190126_world_packets_via_aws_regions2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1036" data-original-width="1600" height="414" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjbte3NwFqWOtW5CGd2pLfIxVF2ERWELeOSP8oesqeXAbqcrhVKyZXYjSeb4w9-_1ipCE4gnGS2n1qWLLLeN6o7y3YMJg2trfmq4goQOxbAaD5ofB5Y8R-XGPWFxwwdGlIZ8Cq-2Df1ucI/s640/20190126_world_packets_via_aws_regions2.png" width="640" /></a></div>
through the following Amazon regions<br />
<h3>
AWS Regions</h3>
<div>
Propagation order</div>
<div>
<a class="region available-region" data-region-id="us-east-1" href="https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top">US East (N. Virginia)</a><a class="region available-region" data-region-id="us-east-1" href="https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="ca-central-1" href="https://ca-central-1.console.aws.amazon.com/ec2/v2/home?region=ca-central-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: inline !important; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top">Canada (Central)</a><span class="region current-region" style="border-left: 3px solid rgb(255 , 153 , 17); color: #444444; cursor: default; display: block; font-family: "helvetica neue" , "roboto" , "arial" , "droid sans" , sans-serif; font-size: 13px; font-weight: bold; margin: 15px 0px; padding: 0px 8px; white-space: nowrap;">US East (Ohio)</span><a class="region available-region" data-region-id="us-west-1" href="https://us-west-1.console.aws.amazon.com/ec2/v2/home?region=us-west-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top">US West (N. California)</a><a class="region available-region" data-region-id="us-west-2" href="https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top">US West (Oregon)</a><a class="region available-region" data-region-id="us-west-2" href="https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="ap-northeast-1" href="https://ap-northeast-1.console.aws.amazon.com/ec2/v2/home?region=ap-northeast-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: inline !important; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top">Asia Pacific (Tokyo)</a><a class="region available-region" data-region-id="us-west-2" href="https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="ap-northeast-2" href="https://ap-northeast-2.console.aws.amazon.com/ec2/v2/home?region=ap-northeast-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top">Asia Pacific (Seoul)</a><a class="region available-region" data-region-id="ap-northeast-2" href="https://ap-northeast-2.console.aws.amazon.com/ec2/v2/home?region=ap-northeast-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="ap-southeast-1" href="https://ap-southeast-1.console.aws.amazon.com/ec2/v2/home?region=ap-southeast-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: inline !important; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top"><br />Asia Pacific (Singapore)</a><a class="region available-region" data-region-id="ap-southeast-2" href="https://ap-southeast-2.console.aws.amazon.com/ec2/v2/home?region=ap-southeast-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top">Asia Pacific (Sydney) - skip</a><a class="region available-region" data-region-id="ap-northeast-1" href="https://ap-northeast-1.console.aws.amazon.com/ec2/v2/home?region=ap-northeast-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="ca-central-1" href="https://ca-central-1.console.aws.amazon.com/ec2/v2/home?region=ca-central-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: inline !important; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top"><br /></a><a class="region available-region" data-region-id="ap-south-1" href="https://ap-south-1.console.aws.amazon.com/ec2/v2/home?region=ap-south-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: inline !important; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top">Asia Pacific (Mumbai)</a><a class="region available-region" data-region-id="eu-central-1" href="https://eu-central-1.console.aws.amazon.com/ec2/v2/home?region=eu-central-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top">EU (Frankfurt)</a><a class="region available-region" data-region-id="eu-central-1" href="https://eu-central-1.console.aws.amazon.com/ec2/v2/home?region=eu-central-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="eu-west-3" href="https://eu-west-3.console.aws.amazon.com/ec2/v2/home?region=eu-west-3#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top">EU (Paris)</a><a class="region available-region" data-region-id="eu-west-2" href="https://eu-west-2.console.aws.amazon.com/ec2/v2/home?region=eu-west-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="eu-north-1" href="https://eu-north-1.console.aws.amazon.com/ec2/v2/home?region=eu-north-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: inline !important; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top"><br />EU (Stockholm) - skip</a><a class="region available-region" data-region-id="eu-west-2" href="https://eu-west-2.console.aws.amazon.com/ec2/v2/home?region=eu-west-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top">EU (London)</a><a class="region available-region" data-region-id="eu-west-2" href="https://eu-west-2.console.aws.amazon.com/ec2/v2/home?region=eu-west-2#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="eu-west-1" href="https://eu-west-1.console.aws.amazon.com/ec2/v2/home?region=eu-west-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top">EU (Ireland)</a><a class="region available-region" data-region-id="eu-north-1" href="https://eu-north-1.console.aws.amazon.com/ec2/v2/home?region=eu-north-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"></a><a class="region available-region" data-region-id="sa-east-1" href="https://sa-east-1.console.aws.amazon.com/ec2/v2/home?region=sa-east-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: inline !important; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none;" target="_top">South America (São Paulo)</a> - skip<a class="region available-region" data-region-id="eu-north-1" href="https://eu-north-1.console.aws.amazon.com/ec2/v2/home?region=eu-north-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"><br /></a><a class="region available-region" data-region-id="sa-east-1" href="https://sa-east-1.console.aws.amazon.com/ec2/v2/home?region=sa-east-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"><br /></a><a class="region available-region" data-region-id="sa-east-1" href="https://sa-east-1.console.aws.amazon.com/ec2/v2/home?region=sa-east-1#Home:" style="border-left: 3px solid transparent; color: #444444; display: block; font-family: "Helvetica Neue", Roboto, Arial, "Droid Sans", sans-serif; font-size: 13px; margin: 15px 0px; padding: 0px 8px; text-decoration-line: none; white-space: nowrap;" target="_top"><br /></a></div>
<br />Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-83900995001026868212018-11-08T05:22:00.001-08:002019-07-28T07:07:23.502-07:00Bluetooth 5 LE Mobile Rest client on iOS 12 with a backend JAX-RS JPA 2 Java cloud service on AWSBluetooth 4 LE Mobile Rest client on iOS 12/3 with a backend JAX-RS JPA 2 Java cloud service on AWS<br />
# Overview<br />
- Full phone metric capture - GPS, Gravity, Accell, Orientation, Magnetic field, pressure, altitude, ambience, heading<br />
- Additional device capture - 2 HRM bluetooth LE metrics - 2A37<br />
- Derived metric capture - Geohash, speed<br />
<br />
# JIRAs<br />
Older system : https://obrienlabs.atlassian.net/projects/BIOM/issues/BIOM-2?filter=allopenissues<br />
https://github.com/obrienlabs/biometric/issues<br />
<br />
# Features<br />
- Live google maps/HRM browser based user tracking<br />
- KML google earth extract<br />
- CSV extract<br />
- XML extract<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivFvM7ne32IlfCOHhvt9qvh9B3fNpEJ_l8Wp07iDKfTd90cNiayCkaZCiPuGG5yE-_XrvXJmavBq-mpr0TYq04J35y-OtNaSWvS1bgHNLtHBtC2TpmJpdTbfLb56P6mA0hkrvkCYqZrwlW/s1600/201811075_roller_google_earth.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivFvM7ne32IlfCOHhvt9qvh9B3fNpEJ_l8Wp07iDKfTd90cNiayCkaZCiPuGG5yE-_XrvXJmavBq-mpr0TYq04J35y-OtNaSWvS1bgHNLtHBtC2TpmJpdTbfLb56P6mA0hkrvkCYqZrwlW/s320/201811075_roller_google_earth.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9Asbij7eyfGhz7CbixsX2wVqw5TgeRN9qkkv-kHouNYtGcMJfuohVq5p3JFC23vc-Ji61VFOXcRxp8TZp6V3VZQp3qdaFovAh8vSShpsxX4rDcWuc1ffQYqvFNRdBpZRersJ3EPfuuSME/s1600/_2014-04-10+08.26.43.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="1136" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9Asbij7eyfGhz7CbixsX2wVqw5TgeRN9qkkv-kHouNYtGcMJfuohVq5p3JFC23vc-Ji61VFOXcRxp8TZp6V3VZQp3qdaFovAh8vSShpsxX4rDcWuc1ffQYqvFNRdBpZRersJ3EPfuuSME/s320/_2014-04-10+08.26.43.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhO8TDhF-ToQmgztx-oaTc9GeVAFdLsfe3oJqomJ__G6w-Zmm5x832K5hPlHjlvQkPkW5Y6-lDbv8VxiQmCvEHHwgIy10mha0x-eZMjfsb5L4QajChksYRaj8U53YRfbs9RQZz9F5y-FeLZ/s1600/_2014-06-24+22.03.33.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1136" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhO8TDhF-ToQmgztx-oaTc9GeVAFdLsfe3oJqomJ__G6w-Zmm5x832K5hPlHjlvQkPkW5Y6-lDbv8VxiQmCvEHHwgIy10mha0x-eZMjfsb5L4QajChksYRaj8U53YRfbs9RQZz9F5y-FeLZ/s320/_2014-06-24+22.03.33.png" width="180" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmwfEa_cUukxEN7VyI30oO1ED96lfJjCZxrPPQAPoaATM-Rn1WfBIQ_Ed11k7ipaO55hLHgK06zaN0D5glO7v2JFfaCISMEDaiTk9BoUIYNvSqXGMgBdJF0tr4tQOP9t7_Qx9TBsxSSqk4/s1600/IMG_20171203_1832057.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1067" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmwfEa_cUukxEN7VyI30oO1ED96lfJjCZxrPPQAPoaATM-Rn1WfBIQ_Ed11k7ipaO55hLHgK06zaN0D5glO7v2JFfaCISMEDaiTk9BoUIYNvSqXGMgBdJF0tr4tQOP9t7_Qx9TBsxSSqk4/s320/IMG_20171203_1832057.jpg" width="213" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_LleSU6Xd8oW3ShSYSCHv4qGjly6dl6SdjF16M0raWay5bzZI2cwXA_ia-vFKQzaSaU4wa3w5kBqF0b4IfHHPuJqYvXv_y7RJrvfnczSTRixk53yexMqJOGdKA6NIqU1FpahvFcMuRbE4/s1600/Screenshot_20181108-083250.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1067" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_LleSU6Xd8oW3ShSYSCHv4qGjly6dl6SdjF16M0raWay5bzZI2cwXA_ia-vFKQzaSaU4wa3w5kBqF0b4IfHHPuJqYvXv_y7RJrvfnczSTRixk53yexMqJOGdKA6NIqU1FpahvFcMuRbE4/s320/Screenshot_20181108-083250.png" width="213" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtgI7t0Nl-VxhacCvOiq_Qf58Dzo03MmylwRxVE-e-9iXtFC-AivYteQbyAKZy76Mukxk6NOD1pkCXRfqIYHXzNmm5deYjujC3Q0RTvEfDjs0UJC7mM2atrQ3QFfxpkqzvRw2vilC-H73J/s1600/IMG_20181020_2017362.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1280" data-original-width="853" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtgI7t0Nl-VxhacCvOiq_Qf58Dzo03MmylwRxVE-e-9iXtFC-AivYteQbyAKZy76Mukxk6NOD1pkCXRfqIYHXzNmm5deYjujC3Q0RTvEfDjs0UJC7mM2atrQ3QFfxpkqzvRw2vilC-H73J/s320/IMG_20181020_2017362.jpg" width="213" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfeR9LUeFUUBMgvPMadTgHdw_eAZL_cEn8KqAauVVWfsfEBiTh5KhhaiQLyxm5bskUZik9ItagXGJUmNJ7A8pP8l-rnwna77QsoOkf9_KnOkQ9H81EzTMd5d7ZDeBVLbnNr-d2ASK8zOJ7/s1600/IMG_20170822_2214496.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1067" data-original-width="1600" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfeR9LUeFUUBMgvPMadTgHdw_eAZL_cEn8KqAauVVWfsfEBiTh5KhhaiQLyxm5bskUZik9ItagXGJUmNJ7A8pP8l-rnwna77QsoOkf9_KnOkQ9H81EzTMd5d7ZDeBVLbnNr-d2ASK8zOJ7/s320/IMG_20170822_2214496.jpg" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRgYBkqiZ13EbuPdOwu8q1uC7O9joXG_01Gx9qHe7VopQbMWNANRcax0H0k_pBPXy5NQU3-oO3nkk7POvywhEN1niNc1myey2MtE1z3P1vYQhVlED9Ai2DpZ9XYrihlBtW3y1zRB2i1kIj/s1600/20150904_rollerblade_keyhole2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRgYBkqiZ13EbuPdOwu8q1uC7O9joXG_01Gx9qHe7VopQbMWNANRcax0H0k_pBPXy5NQU3-oO3nkk7POvywhEN1niNc1myey2MtE1z3P1vYQhVlED9Ai2DpZ9XYrihlBtW3y1zRB2i1kIj/s320/20150904_rollerblade_keyhole2.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizqTbzBLD3P3ulRM4BYVXZe1vpvuLBJAgVmCr70h58n0bjWBTSDn4pEw-m9CEOsxKkK9GF37k5R7irkFqLUVFIbHVS2f_n09PAYGR72KOcnAYF9_WOkn73ezvjEsSMs9I1cA7ce35hku_J/s1600/201311_goodlife_run_10_30_15_24.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="904" data-original-width="1600" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizqTbzBLD3P3ulRM4BYVXZe1vpvuLBJAgVmCr70h58n0bjWBTSDn4pEw-m9CEOsxKkK9GF37k5R7irkFqLUVFIbHVS2f_n09PAYGR72KOcnAYF9_WOkn73ezvjEsSMs9I1cA7ce35hku_J/s320/201311_goodlife_run_10_30_15_24.jpg" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj52mi7WNHC63wDy9Bn1HU61BJBEAKFZDdjR1y4pBevN_45TLX6r_TOQ8rV6hRJd5s7KBREr8WaEaNzP46X2gS45VLhT_5A2hv__CyvgUYmdHyu8hvIJw6WB39TniaZbRxaHZHyVIK6N5sB/s1600/20181004_run_dual_IMG_0167.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1334" data-original-width="750" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj52mi7WNHC63wDy9Bn1HU61BJBEAKFZDdjR1y4pBevN_45TLX6r_TOQ8rV6hRJd5s7KBREr8WaEaNzP46X2gS45VLhT_5A2hv__CyvgUYmdHyu8hvIJw6WB39TniaZbRxaHZHyVIK6N5sB/s320/20181004_run_dual_IMG_0167.PNG" width="179" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDd0y8DtDVi67zTApPviOPY7p2l8rkiOlBg4nUrYUW6uvwXO7m0ZFDQ1b6ndpPRdJN6EdON41REYbKKu-G4KZF8LL2EeRNiHdR9XY5IEGgRdWbp1rZiJoOWSacT6wlHhBBgJ45EidW9ine/s1600/2015-11-29+12.27.42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1136" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDd0y8DtDVi67zTApPviOPY7p2l8rkiOlBg4nUrYUW6uvwXO7m0ZFDQ1b6ndpPRdJN6EdON41REYbKKu-G4KZF8LL2EeRNiHdR9XY5IEGgRdWbp1rZiJoOWSacT6wlHhBBgJ45EidW9ine/s320/2015-11-29+12.27.42.png" width="180" /></a></div>
<br />Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-81116462209539343762017-09-18T19:42:00.007-07:002019-10-05T06:18:12.609-07:00Docker and Kubernetes DevOps<br />
<h3>
see <a href="http://wiki.obrienlabs.cloud/display/DEV/Kubernetes+Developer+Guide">http://wiki.obrienlabs.cloud/display/DEV/Kubernetes+Developer+Guide</a></h3>
<h3>
NAT settings for OSX Fusion 8.5</h3>
<br />
<div class="p1">
<span class="s1">sudo vi /Library/Preferences/VMware\ Fusion/vmnet8/nat.conf</span></div>
<div class="p2">
<span class="s1"></span></div>
<div class="p1">
<span class="s1">[incomingtcp]</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1"># Use these with care - anyone can enter into your VM through these...</span></div>
<div class="p1">
<span class="s1"># The format and example are as follows:</span></div>
<div class="p1">
<span class="s1">#<external port number> = <VM's IP address>:<VM's port number></span></div>
<div class="p1">
<span class="s1">#8080 = 172.16.3.128:80</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">8880 = 192.168.241.134:8880</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">$ sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --stop</span></div>
<div class="p1">
<span class="s1">Stopped DHCP service on vmnet1</span></div>
<div class="p1">
<span class="s1">Disabled hostonly virtual adapter on vmnet1</span></div>
<div class="p1">
<span class="s1">Stopped DHCP service on vmnet8</span></div>
<div class="p1">
<span class="s1">Stopped NAT service on vmnet8</span></div>
<div class="p1">
<span class="s1">Disabled hostonly virtual adapter on vmnet8</span></div>
<div class="p1">
<span class="s1">Stopped all configured services on all networks</span></div>
<div class="p1">
<span class="s1">$ sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --start</span></div>
<div class="p1">
<span class="s1">Enabled hostonly virtual adapter on vmnet1</span></div>
<div class="p1">
<span class="s1">Started DHCP service on vmnet1</span></div>
<div class="p1">
<span class="s1">Started NAT service on vmnet8</span></div>
<div class="p1">
<span class="s1">Enabled hostonly virtual adapter on vmnet8</span></div>
<div class="p1">
<span class="s1">Started DHCP service on vmnet8</span></div>
<div class="p1">
<span class="s1">Started all configured services on all networks</span></div>
<div class="p1">
<span class="s1">$ curl 127.0.0.1:8880</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #173b79}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #173b79; min-height: 13.0px}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">{"id":"v1","type":"apiVersion","links":{"accounts":"http:\/\/127.0.0.1:8880\/v1\/accounts","agents":"http:\/\/127.0.0.1:8880\/v1\/agents","apiKeys":"http:\/\/127.0.0.1:8880\/</span><br />
<span class="s1"><br /></span>
<br />
<h3>
<span class="s1">Running a VNC ubuntu VM inside the namespace</span></h3>
<div class="p1">
<a href="https://wiki.onap.org/display/DW/ONAP+on+Kubernetes#ONAPonKubernetes-QuickstartInstallation">https://wiki.onap.org/display/DW/ONAP+on+Kubernetes#ONAPonKubernetes-QuickstartInstallation</a></div>
<div class="p2">
<span class="s1"># </span><span class="s2"><b>for</b></span><span class="s3"> </span><span class="s1">root just run the following line and skip to next section</span></div>
<div class="p3">
<span class="s4">curl https:</span><span class="s1">//releases.rancher.com/install-docker/1.12.sh | sh</span></div>
<div class="p2">
<span class="s1"># when running as non-root (ubuntu) run the following and logout/log back in</span></div>
<div class="p2">
<span class="s1">sudo usermod -aG docker ubuntu</span></div>
<div class="p4">
<br /></div>
<div class="p5">
Also we are using this vnc-portal in production - been vetted for 4 months</div>
<div class="p4">
<br /></div>
<div class="p5">
<a href="https://git.onap.org/oom/tree/kubernetes/portal/values.yaml">https://git.onap.org/oom/tree/kubernetes/portal/values.yaml</a></div>
<div class="p4">
<br /></div>
<div class="p5">
his docker image is on</div>
<div class="p4">
<br /></div>
<div class="p5">
<a href="https://hub.docker.com/r/dorowu/ubuntu-desktop-lxde-vnc/">https://hub.docker.com/r/dorowu/ubuntu-desktop-lxde-vnc/</a></div>
<div class="p4">
<br /></div>
<div class="p6">
<span class="s1">docker run -it --rm -p 6080:80 dorowu/ubuntu-desktop-lxde-vnc</span></div>
<div class="p7">
<span class="s1">access it with</span></div>
<div class="p7">
<span class="s1"><a href="http://127.0.0.1:6080/">http://127.0.0.1:6080/<span class="s5"></span></a></span></div>
<div class="p8">
<span class="s1"></span><br /></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 20.0px; font: 14.0px Consolas; color: #000000; -webkit-text-stroke: #000000; background-color: #ffffff}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 20.0px; font: 14.0px Consolas; color: #008200; -webkit-text-stroke: #008200; background-color: #ffffff}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; -webkit-text-stroke: #333333; min-height: 14.0px}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; -webkit-text-stroke: #333333}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 19.0px; font: 16.0px Consolas; color: #333333; -webkit-text-stroke: #333333; background-color: #f8f8f8}
p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 18.0px; font: 16.0px 'Helvetica Neue'; color: #22b8eb; -webkit-text-stroke: #22b8eb; background-color: #ffffff}
p.p8 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 18.0px; font: 16.0px 'Helvetica Neue'; color: #22b8eb; -webkit-text-stroke: #22b8eb; background-color: #ffffff; min-height: 18.0px}
span.s1 {font-kerning: none}
span.s2 {font-kerning: none; color: #336699; -webkit-text-stroke: 0px #336699}
span.s3 {font-kerning: none; color: #333333; -webkit-text-stroke: 0px #333333}
span.s4 {font-kerning: none; color: #000000; -webkit-text-stroke: 0px #000000}
span.s5 {font: 16.0px 'Helvetica Neue'; font-kerning: none; color: #22b8eb; -webkit-text-stroke: 0px #22b8eb}
</style>
</span><br />
<div class="p7">
<span class="s1">if you ctrl-c then the docker will exit - it is stateless</span></div>
</div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com5tag:blogger.com,1999:blog-3810253236812062548.post-81246015676469367752017-03-19T05:39:00.000-07:002017-04-05T20:17:27.820-07:00Optimizing your Macbook Pro 15 2016 for development - minimize touchbar interference<br />
This review is for the <a href="https://en.wikipedia.org/wiki/MacBook_Pro#Technical_specifications_4">Late 2016 Macbook Pro 15 inch</a> with the upgraded 2.9GHz i7-6920HQ, the 1TB SSD and the ATI Radeon Pro 460 4Gb and the touchbar.<br />
<br />
Sometimes I get asked why I just don't use a high end Lenovo (with a Xeon and 64G ram). The issue is I also need iOS development via XCode. So for now I need to put up with 16G ram until 32G comes out.<br />
<br />
Most hard core users of the macbook pro will benefit from the workarounds below - especially working around the new touchbar - by mostly disabling it.<br />
<br />
After working with the laptop for home and office development - I can say that it is a better machine than the previous generation. The improvements outweigh the discrepancies at a bit higher price.<br />
<br />
The HD is really fast - faster than either my raid0 4 drive Samsung 850 pro or my Intel 750. The screen is brighter, the CPU is 23% faster under full load (12% because of clock speed), the GPU is twice as fast.<br />
<br />
The keys are louder and bit clunky, but the biggest issue is the touchbar - it must be disabled!.<br />
As soon as you start using the machine in its default config - your finger will slide off the delete key - where it will start siri (all the time). The fix for the touchbar is to remove all the icons by default and only enable the function keys for specific applications. The rest of the time only the escape key shows. When the mac touchbar is disabled like this then when your fingers - just touch above the number keys - you wont invoke the touchbar.<br />
<br />
<h3>
Summary</h3>
<div>
0 - The F4 key rarely works - this is a Safari issue though<br />
<br />
1 - Globally disable the touchbar except for the escape key for all non-development apps<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh01CCj-Sf3TCHFWXGSjsBaxoL3xz86kF6ZB9_RX-VLGgc7dozRy64x99XwXSFzV7f_naSBYIRMSm2gK5MQoG3VohlTkVlyNw_cyiHMroQyHwVSqFgUlm_jG0guP06mqm0eH5TFPo6jmZje/s1600/IMG_20170309_1502228.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh01CCj-Sf3TCHFWXGSjsBaxoL3xz86kF6ZB9_RX-VLGgc7dozRy64x99XwXSFzV7f_naSBYIRMSm2gK5MQoG3VohlTkVlyNw_cyiHMroQyHwVSqFgUlm_jG0guP06mqm0eH5TFPo6jmZje/s320/IMG_20170309_1502228.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpb_H2h984fI0lj_w-zhdgv-wEzSgRrvsM82m55qi2q46NbVtmf_X_Ll_FdUKbu7LWhbLBLAVE0enzqajl3FbRdtL9j_h77IsFCsZG17rqj6H_7E3bt8-qiJn-OwRrReQuRe752RzvKQ6Y/s1600/IMG_20170309_1502289.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpb_H2h984fI0lj_w-zhdgv-wEzSgRrvsM82m55qi2q46NbVtmf_X_Ll_FdUKbu7LWhbLBLAVE0enzqajl3FbRdtL9j_h77IsFCsZG17rqj6H_7E3bt8-qiJn-OwRrReQuRe752RzvKQ6Y/s320/IMG_20170309_1502289.jpg" width="320" /></a></div>
<br />
<br /></div>
<div>
2 - Enable the function keys for development apps like Eclipse and IntelliJ</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIA0g2ZZKoiPWraLzilMOUVG3QN6OthIxAgIl1OL_zcx5CCVnL1I8x-21aASAwftHs1PyuGG8LcSfv8IzOt-IoF612Pu1091yzRlK483Csnykab_GFNqbQfnzRJ38kwl36dr83PQ41nsQn/s1600/Screenshot+2017-03-19+09.25.53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIA0g2ZZKoiPWraLzilMOUVG3QN6OthIxAgIl1OL_zcx5CCVnL1I8x-21aASAwftHs1PyuGG8LcSfv8IzOt-IoF612Pu1091yzRlK483Csnykab_GFNqbQfnzRJ38kwl36dr83PQ41nsQn/s640/Screenshot+2017-03-19+09.25.53.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2Ryy8CAwoJJKjROFLljSOQDfkSCScqMOmtw6rfTt4FmQsnZpL9T-t5tnO85_VE7BkJt2a-1ySsuLwWLQEzVYtgYdyVQJM_jJAr-XceZ7ydDTrd4Q0GTsHqD8JMi7tz-qHbfR_jGqN6MDu/s1600/Screenshot+2017-03-19+09.26.08.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="96" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2Ryy8CAwoJJKjROFLljSOQDfkSCScqMOmtw6rfTt4FmQsnZpL9T-t5tnO85_VE7BkJt2a-1ySsuLwWLQEzVYtgYdyVQJM_jJAr-XceZ7ydDTrd4Q0GTsHqD8JMi7tz-qHbfR_jGqN6MDu/s640/Screenshot+2017-03-19+09.26.08.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMcp9oOBBbduW5bBR3PC2MQpYVo1b8gp_rsOsIWflGqjBTpmkMujT4ctn-eUXkmx7EZDCi3lAXQSKE2014s1Xv_xsfFgcBAB5PjC8lrE9PpOwgd2d1-VzG7nPJaw7-pJqjAcqf2SzrqUyg/s1600/Screenshot+2017-03-19+09.25.20.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="347" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMcp9oOBBbduW5bBR3PC2MQpYVo1b8gp_rsOsIWflGqjBTpmkMujT4ctn-eUXkmx7EZDCi3lAXQSKE2014s1Xv_xsfFgcBAB5PjC8lrE9PpOwgd2d1-VzG7nPJaw7-pJqjAcqf2SzrqUyg/s400/Screenshot+2017-03-19+09.25.20.png" width="400" /></a></div>
<div>
<br /></div>
<br />
<h3>
Macbook 2016 Improvements</h3>
The 2016 i7-6920HZ CPU runs 23% faster than the my late 2013 i7-4960 (12% of this is due to a clock speed jump from 2.6 to 2.9 GHz)<br />
The GPU is faster - although this is hard to measure because I went from an NVidia 750GT to an ATI 460<br />
<h3>
Macbook 2016 Issues</h3>
The keyboard is lower quality than the 2015 model<br />
The GPU is ATI instead of NVidia - which locks us out of must CUDA based libraries like TensorFlow leaving us with OpenCL<br />
The touchbar causes multiple issues - the biggest one is launching siri or changing sound/screen inadvertently by accidentally touching the bar above the number row.<br />
<br />
<h3>
Minimize Macbook Pro 2016 Touchbar effects</h3>
The first thing you will notice when using the 2016 macbook will be how the touchbar causes issues north of the delete key when your fingers slide off the northbound part of the key.<br />
The goal is to disable the touchbar for most apps leaving just the escape key.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqYPYhf3T_ixgndwRxrkO91CPeSmn75Wo5exBFWNz0QwlSL8YtsxrdrzqGnh7CnpYqSjZsHwjAAJGj6utT8X87wXvGXDBHEFSAwf__1TaOPy3QhUG1iwfVSZXSEM0qeZWy9yYf7vpaEdm9/s1600/Screenshot+2017-03-19+09.24.30.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="347" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqYPYhf3T_ixgndwRxrkO91CPeSmn75Wo5exBFWNz0QwlSL8YtsxrdrzqGnh7CnpYqSjZsHwjAAJGj6utT8X87wXvGXDBHEFSAwf__1TaOPy3QhUG1iwfVSZXSEM0qeZWy9yYf7vpaEdm9/s400/Screenshot+2017-03-19+09.24.30.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_JuVFijusckdoo0iw6K0bvYgiN2CkOmVSQfDcwDc3peAyWSCpZvjVlHd_LTl6Epn0oZLo3ENRupqyRHfTBGp0tXzsoXkmSEIYRIHm2rMp1SPSlTYacevKu1sASSZOc40QuoVbwi3Dg59H/s1600/Screenshot+2017-03-19+09.24.51.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_JuVFijusckdoo0iw6K0bvYgiN2CkOmVSQfDcwDc3peAyWSCpZvjVlHd_LTl6Epn0oZLo3ENRupqyRHfTBGp0tXzsoXkmSEIYRIHm2rMp1SPSlTYacevKu1sASSZOc40QuoVbwi3Dg59H/s640/Screenshot+2017-03-19+09.24.51.png" width="640" /></a></div>
<br />
<br />
For development apps that require function keys like the F3/F4 (navigation)and F5/F6/F7/F8 (debug) functions in Eclipse - enable just the function key view.<br />
<br />
Here in eclipse we frequently need F3 to load a class and F4 to get the inheritance heirarchy<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBbgb-KZGQOww4EuXuKcrNX57AhbiL1yEEZiZb7gdKLkuO13F-oaQ6EaA7YEblD34NVQs8OfpoJpUme0ZrO5y0WX6xC8OUi-VD2Xf9ufXh35zVtdDZGZsuTEQuMidwJZzy1fteZgakEmGU/s1600/Screenshot+2017-03-19+09.38.59.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="187" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBbgb-KZGQOww4EuXuKcrNX57AhbiL1yEEZiZb7gdKLkuO13F-oaQ6EaA7YEblD34NVQs8OfpoJpUme0ZrO5y0WX6xC8OUi-VD2Xf9ufXh35zVtdDZGZsuTEQuMidwJZzy1fteZgakEmGU/s400/Screenshot+2017-03-19+09.38.59.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
During a debug session we use F5 to enter, F6 to run, F7 to skip and F8 to resume<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirJYy8SEnosvj7t5FVJXQla33sXseyg40Sp88UYFkwldZp625o0qj42cSQs1vhAPCAJ53hcI48osC8QO4vb-9j_mMoXwg-JgrFZ76CeUAEtmQlOHQHvX2XeRXhPQClFOwzZmXkanNq-RN7/s1600/Screenshot+2017-03-19+09.39.42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirJYy8SEnosvj7t5FVJXQla33sXseyg40Sp88UYFkwldZp625o0qj42cSQs1vhAPCAJ53hcI48osC8QO4vb-9j_mMoXwg-JgrFZ76CeUAEtmQlOHQHvX2XeRXhPQClFOwzZmXkanNq-RN7/s640/Screenshot+2017-03-19+09.39.42.png" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0SWY76ygVO1sgAEn3ldxVPzn9xgUQcacFVi5g1dtMO01grjQb1exaB5iz1mxUs-8ckO6Mp-w90Rx2XerzN2WLwc-oqUpiIaRK6rrPnDxlXmDXZJpwbmUWktPej3aK6zIJa2L9NnBu-Zc7/s1600/Screenshot+2017-03-02+19.51.55.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="133" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0SWY76ygVO1sgAEn3ldxVPzn9xgUQcacFVi5g1dtMO01grjQb1exaB5iz1mxUs-8ckO6Mp-w90Rx2XerzN2WLwc-oqUpiIaRK6rrPnDxlXmDXZJpwbmUWktPej3aK6zIJa2L9NnBu-Zc7/s400/Screenshot+2017-03-02+19.51.55.png" width="400" /></a></div>
<br />
<h3>
CPU and GPU performance</h3>
<div>
On average GPU speed for the 2016 macbook pro (top model) is 12x the cpu speed (all 4 cores)</div>
<div>
Surprisingly the embedded GPU in the CPU runs slower than the embedded Iris GPU in my old 2013 macbook pro.</div>
<h4>
OpenCL</h4>
<div style="text-align: center;">
Single Core CPU = 13</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgupkgzNW1_Fr50PCInFLwqHH3l89PGCE8gixA506TIiihc9MkTRFUfU8_UYxIvkYe6s8H77CrHjGmhGaSTnvJQM6pYcS70xkvMRkk491Ep-nVi0XGxIX8gJ4VLnioo8gneT5ZeZktMxpmB/s1600/Screenshot+2017-03-19+09.22.28.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="402" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgupkgzNW1_Fr50PCInFLwqHH3l89PGCE8gixA506TIiihc9MkTRFUfU8_UYxIvkYe6s8H77CrHjGmhGaSTnvJQM6pYcS70xkvMRkk491Ep-nVi0XGxIX8gJ4VLnioo8gneT5ZeZktMxpmB/s640/Screenshot+2017-03-19+09.22.28.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Multicore 4 CPU = 50</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoAQMuEFjw3KLEfS20LKVgRFxrmIjUl-iYlj0MPi1TTjdXi5NaYM50Sc_eVvSwbGp-I18pW94BxEafyKyZiBJFEbxcLiWpKpcxTbDerwn-35htuSL5hPdo6IfM8a6uyRS363K-gAB6_CtE/s1600/Screenshot+2017-03-19+09.22.48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoAQMuEFjw3KLEfS20LKVgRFxrmIjUl-iYlj0MPi1TTjdXi5NaYM50Sc_eVvSwbGp-I18pW94BxEafyKyZiBJFEbxcLiWpKpcxTbDerwn-35htuSL5hPdo6IfM8a6uyRS363K-gAB6_CtE/s640/Screenshot+2017-03-19+09.22.48.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Embedded HD 530 GPU = 128</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSN5g63fCZ8BhZmvh3tU0kkVXWeEu-mw2EEzSX-wuuj_Ex6bjYyYulIuTOU9lxHEQmqSBQvKPwd_0o1GJxVplsKjtJz_PZMJqtuvSi0dzbkHsnbr3WnyqJP68kkPhzTyCCibKJzlSB8_DH/s1600/Screenshot+2017-03-19+09.23.07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="402" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSN5g63fCZ8BhZmvh3tU0kkVXWeEu-mw2EEzSX-wuuj_Ex6bjYyYulIuTOU9lxHEQmqSBQvKPwd_0o1GJxVplsKjtJz_PZMJqtuvSi0dzbkHsnbr3WnyqJP68kkPhzTyCCibKJzlSB8_DH/s640/Screenshot+2017-03-19+09.23.07.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
ATI HD 460 GPU = 505</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkjAXCEHPuKfQiVxGi8cwQe5FSLME46W5HGVnQF-u2OY_8i3apBwDoTKZBr5nvZGRd3i8psh4zY0Psjlg6JqyFvFlcO5Pu80sUl5MZWF6xZSHhC7bYDIsl5bzUx6I4n3q1FJuKGv97IeD3/s1600/Screenshot+2017-03-19+09.23.22.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkjAXCEHPuKfQiVxGi8cwQe5FSLME46W5HGVnQF-u2OY_8i3apBwDoTKZBr5nvZGRd3i8psh4zY0Psjlg6JqyFvFlcO5Pu80sUl5MZWF6xZSHhC7bYDIsl5bzUx6I4n3q1FJuKGv97IeD3/s640/Screenshot+2017-03-19+09.23.22.png" width="640" /></a></div>
<br />
<h4>
Java</h4>
<h3>
Connectors</h3>
<div>
I'll get into theUSB-C connectors shortly</div>
<h3>
Cases</h3>
Get the Thule case for your Macbook Pro 15 <a href="https://www.thule.com/en-us/ca/computer-phone-cases/laptop-cases/thule-gauntlet-30-macbook-pro-sleeve-15-_-tl_85854234856">https://www.thule.com/en-us/ca/computer-phone-cases/laptop-cases/thule-gauntlet-30-macbook-pro-sleeve-15-_-tl_85854234856</a><br />
<br />
<h3>
Noise</h3>
The new keyboard is louder - try to type quieter in an office settingMichael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-46741829055081540462017-02-02T19:17:00.002-08:002017-03-19T05:33:49.017-07:00Deep Learning Experiments<h4>
Goal: </h4>
Ramp up on Deep Learning techniques by building a basic ANN to classify radar images for example<br />
<br />
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu0dGew2WIPozKmyW3qHH3IzYFqZhbxN6ThJtbBbEyUw6WAk2DbmVL36-aZU_Ty17VV_K_mLhKbQb_-Ivc90l-kKYi7974vtA1Z-THxnNa8MigFghmwgUg2KG4MTFDU6x-0e3Dxda8I_NE/s1600/neo4j_4096_nodes_49152rels_unsorted.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu0dGew2WIPozKmyW3qHH3IzYFqZhbxN6ThJtbBbEyUw6WAk2DbmVL36-aZU_Ty17VV_K_mLhKbQb_-Ivc90l-kKYi7974vtA1Z-THxnNa8MigFghmwgUg2KG4MTFDU6x-0e3Dxda8I_NE/s320/neo4j_4096_nodes_49152rels_unsorted.png" width="320" /></a></div>
Get some background in deep learning (a specialized subset of machine learning concentrated on the "<strong>Greedy Layer-Wise Unsupervised Pretraining procedure"</strong> - <a href="http://www.cs.toronto.edu/~fritz/absps/ncfast.pdf" style="color: #3572b0; text-decoration: none;">Hinton, 2006 - University of Toronto</a>) and the <strong>"Long Short-Term Memory model</strong> - <a href="http://people.idsia.ch/~juergen/lstm2003tutorial.pdf" style="color: #3572b0; text-decoration: none;">Hochreiter and Schmidhuber 1997</a>".</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px;">
<span style="font-size: 20px;">Design/Theory Preparation</span></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px;">
Get the following Nov 2016 MIT book from <a data-mce-href="https://www.amazon.ca/Deep-Learning-Ian-Goodfellow/dp/0262035618/ref=sr_1_1?ie=UTF8&qid=1486501444&sr=8-1&keywords=deep+learning" href="https://www.amazon.ca/Deep-Learning-Ian-Goodfellow/dp/0262035618/ref=sr_1_1?ie=UTF8&qid=1486501444&sr=8-1&keywords=deep+learning" style="color: #3572b0; text-decoration: none;">Amazon</a> by Ian Goodfellow, Yoshua Benglo, Aaron Courville - (one for work, and one for home - as it is usually out of stock). Review your linear transformation and matrix math to prep.</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px;">
<br /></div>
<div style="margin-top: 10px;">
<div class="separator" style="clear: both; color: #333333; font-family: arial, sans-serif; font-size: 14px; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5o7uH0nIJSMdt1gTuvuzEjhD_LHD1szLToxx4LKApJI5Fo7x1UQPDukigmD5oF6T-QPOVPpoKspHEu3CK39b6oywoVeyk0MdHLhGZZCF5mGNzbQpIlU5T2mCzIRb7mXO0D2fC3z0OWrdV/s1600/deeplearning_mit_2016_61DF201HelL._SX373_BO1%252C204%252C203%252C200_.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5o7uH0nIJSMdt1gTuvuzEjhD_LHD1szLToxx4LKApJI5Fo7x1UQPDukigmD5oF6T-QPOVPpoKspHEu3CK39b6oywoVeyk0MdHLhGZZCF5mGNzbQpIlU5T2mCzIRb7mXO0D2fC3z0OWrdV/s320/deeplearning_mit_2016_61DF201HelL._SX373_BO1%252C204%252C203%252C200_.jpg" width="240" /></a></div>
<div style="color: #333333; font-family: arial, sans-serif; font-size: 14px;">
<br /></div>
<div style="color: #333333; font-family: arial, sans-serif; font-size: 14px;">
TesorFlow</div>
<span style="color: #333333; font-family: arial, sans-serif;"><span style="font-size: 14px;"><a href="https://www.tensorflow.org/install/install_mac">https://www.tensorflow.org/install/install_mac</a></span></span></div>
<h4>
References:</h4>
Deep Learning, 2016, Ian Goodfellow, Yoshua Benglo, Aaron Courville<br />
<br />
<a href="http://127.0.0.1:20959/doc/manual/R-intro.html#Related-software-and-documentation">http://127.0.0.1:20959/doc/manual/R-intro.html#Related-software-and-documentation</a><br />
<a href="https://www.microsoft.com/cognitive-services/en-us/sign-up">https://www.microsoft.com/cognitive-services/en-us/sign-up</a><br />
<a href="https://deeplearning4j.org/devguide">https://deeplearning4j.org/devguide</a><br />
<a href="https://openai.com/blog">https://openai.com/blog</a>/<br />
<a href="https://github.com/obriensystems/requests-for-research">https://github.com/obriensystems/requests-for-research</a><br />
<a href="https://deepfield.com/cloud-genome/">https://deepfield.com/cloud-genome/</a><br />
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px;">
<a class="" data-mce-href="http://www.cs.toronto.edu/~fritz/absps/ncfast.pdf" href="http://www.cs.toronto.edu/~fritz/absps/ncfast.pdf" style="color: #3572b0; text-decoration: none;">http://www.cs.toronto.edu/~fritz/absps/ncfast.pdf</a></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px;">
<a class="" data-mce-href="http://people.idsia.ch/~juergen/lstm2003tutorial.pdf" href="http://people.idsia.ch/~juergen/lstm2003tutorial.pdf" style="color: #3572b0; text-decoration: none;">http://people.idsia.ch/~juergen/lstm2003tutorial.pdf</a></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px;">
<a class="" data-mce-href="http://dl.acm.org/citation.cfm?id=1246450&CFID=898083621&CFTOKEN=58541063&qualifier=LU1039002" href="http://dl.acm.org/citation.cfm?id=1246450&CFID=898083621&CFTOKEN=58541063&qualifier=LU1039002" style="color: #3572b0; text-decoration: none;">http://dl.acm.org/citation.cfm?id=1246450&CFID=898083621&CFTOKEN=58541063&qualifier=LU103900</a></div>
<br />
<br />Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-82623091952007219892016-12-10T08:22:00.004-08:002016-12-24T07:28:14.122-08:00Distributed in-memory Cache - Apache Ignite<br />
Apache Ignite is highly performant because it has a deep history of distributed deployment with sql, transactional and locking support. It was recently open sourced to Apache.<br />
<br />
Mixed metal and VM's<br />
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:05:22] Ignite node started OK (id=7352a302)</span></div>
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:05:22] Topology snapshot [ver=1, servers=1, clients=0, CPUs=4, heap=12.0GB]</span></div>
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:05:32] Topology snapshot [ver=2, servers=2, clients=0, CPUs=8, heap=24.0GB]</span></div>
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:05:42] Topology snapshot [ver=3, servers=3, clients=0, CPUs=12, heap=36.0GB]</span></div>
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:05:50] Topology snapshot [ver=4, servers=4, clients=0, CPUs=16, heap=48.0GB]</span></div>
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:05:57] Topology snapshot [ver=5, servers=5, clients=0, CPUs=24, heap=60.0GB]</span></div>
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:06:07] Topology snapshot [ver=6, servers=6, clients=0, CPUs=30, heap=68.0GB]</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1" style="font-family: Courier New, Courier, monospace;">[19:06:16] Topology snapshot [ver=7, servers=7, clients=0, CPUs=36, heap=76.0GB]</span></div>
<br />
Bare Metal NUCs<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigKBGt1eD1lB8WV4Qy-x7ZP1teq8tUJEypsQCYT82WXc1BFbVdnWtz29VQcS542Xtn4vZL669KKg9awAR7BpV9a9Qi0U5bfqldk3N16qb6mGZfUt_5uRwxQAblElfTc_-5tbXyMpx7Lhix/s1600/20161211_apache_ignite_180_three_node_cluster_firewall_off.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="572" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigKBGt1eD1lB8WV4Qy-x7ZP1teq8tUJEypsQCYT82WXc1BFbVdnWtz29VQcS542Xtn4vZL669KKg9awAR7BpV9a9Qi0U5bfqldk3N16qb6mGZfUt_5uRwxQAblElfTc_-5tbXyMpx7Lhix/s640/20161211_apache_ignite_180_three_node_cluster_firewall_off.png" width="640" /></a></div>
<br />
VMware RHEL 7.3 Virtual Machines (bridged mode) across two physical hosts<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7d1W1AP4VfEPvjyKID6drUZq2m7kNMWu9QsV5iyrEMXwAQcpqsypGd9qA3DtXjFlFcDXHeFhDjeRanNF4vQBlHiXuhg1CRMq17KBcKRueumLwsOJ8Pj4xFrp-vgtiugpFv7ikXA7xQaLQ/s1600/Screenshot+2016-12-11+23.52.36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7d1W1AP4VfEPvjyKID6drUZq2m7kNMWu9QsV5iyrEMXwAQcpqsypGd9qA3DtXjFlFcDXHeFhDjeRanNF4vQBlHiXuhg1CRMq17KBcKRueumLwsOJ8Pj4xFrp-vgtiugpFv7ikXA7xQaLQ/s1600/Screenshot+2016-12-11+23.52.36.png" /></a></div>
<br />
<br />
<br />
<div class="p1">
<span class="s1"><a href="https://dzone.com/articles/deadlock-free-transactions-with-apache-ignite">https://dzone.com/articles/deadlock-free-transactions-with-apache-ignite</a></span></div>
<div class="p2">
<span class="s2"><br />
GridGain Enterprise Edition 7.5 corresponds to Apache Ignite 1.5 in February 2016</span></div>
<div class="p1">
<span class="s1"><a href="https://ignite.apache.org/">https://ignite.apache.org/</a></span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p2">
<span class="s2">Release 1.8.0 8 Dec 2016</span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<table cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td class="td1" valign="middle"><div class="p2">
<span class="s2">Amazon Image</span></div>
</td>
<td class="td2" valign="middle"><div class="p1">
<span class="s1"><a href="https://apacheignite.readme.io/docs/docker-deployment#amazon-ec2-deployment">guide</a></span></div>
</td>
<td class="td3" valign="middle"><div class="p2">
<span class="s2">2015-06-25</span></div>
</td>
<td class="td4" valign="middle"><div class="p1">
<span class="s1"><a href="https://console.aws.amazon.com/ec2/home?region=us-west-1#launchAmi=ami-51fb0d15">us-west:ami-51fb0d15</a></span><span class="s3">, <a href="https://console.aws.amazon.com/ec2/home?region=us-east-1#launchAmi=ami-0f824764"><span class="s4">us-east:ami-0f824764</span></a>, <a href="https://console.aws.amazon.com/ec2/home?region=eu-central-1#launchAmi=ami-0a4d7617"><span class="s4">eu-central:ami-0a4d7617</span></a></span></div>
</td>
</tr>
</tbody>
</table>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p1">
<span class="s1"><a href="https://apacheignite.readme.io/docs/docker-deployment#amazon-ec2-deployment">https://apacheignite.readme.io/docs/docker-deployment#amazon-ec2-deployment</a></span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p1">
<span class="s1"><a href="https://ignite.apache.org/use-cases/caching/hibernate-l2-cache.html">https://ignite.apache.org/use-cases/caching/hibernate-l2-cache.html</a></span></div>
<div class="p1">
<span class="s1"><a href="https://imcsummit.org/2016/videos-and-slides/">https://imcsummit.org/2016/videos-and-slides/</a></span></div>
<ul class="ul1">
<li class="li2"><span class="s2">Download Apache Ignite as ZIP archive from <a href="https://ignite.apache.org/"><span class="s4">https://ignite.apache.org/</span></a></span></li>
<li class="li2"><span class="s2">Unzip ZIP archive into the installation folder in your system</span></li>
<li class="li2"><span class="s2">Set IGNITE_HOME environment variable to point to the installation folder and make sure there is no trailing / in the path (this step is optional)</span></li>
</ul>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p1">
<span class="s1"><a href="https://dzone.com/articles/running-microservices-on-top-of-in-memory-data-gri">https://dzone.com/articles/running-microservices-on-top-of-in-memory-data-gri</a></span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p2">
<span class="s2">Maven/Gradle dependencies</span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p2">
<span class="s2"><<b>dependency</b>></span></div>
<div class="p2">
<span class="s2"> <<b>groupId</b>>org.apache.ignite</<b>groupId</b>></span></div>
<div class="p2">
<span class="s2"> <<b>artifactId</b>>ignite-core</<b>artifactId</b>></span></div>
<div class="p2">
<span class="s2"> <<b>version</b>>${ignite.version}</<b>version</b>></span></div>
<div class="p2">
<span class="s2"></<b>dependency</b>></span></div>
<div class="p2">
<span class="s2"><<b>dependency</b>></span></div>
<div class="p2">
<span class="s2"> <<b>groupId</b>>org.apache.ignite</<b>groupId</b>></span></div>
<div class="p2">
<span class="s2"> <<b>artifactId</b>>ignite-spring</<b>artifactId</b>></span></div>
<div class="p2">
<span class="s2"> <<b>version</b>>${ignite.version}</<b>version</b>></span></div>
<div class="p2">
<span class="s2"></<b>dependency</b>></span></div>
<div class="p2">
<span class="s2"><<b>dependency</b>></span></div>
<div class="p2">
<span class="s2"> <<b>groupId</b>>org.apache.ignite</<b>groupId</b>></span></div>
<div class="p2">
<span class="s2"> <<b>artifactId</b>>ignite-indexing</<b>artifactId</b>></span></div>
<div class="p2">
<span class="s2"> <<b>version</b>>${ignite.version}</<b>version</b>></span></div>
<div class="p3">
<span class="s2"></span><br /></div>
<div class="p2">
<span class="s2"></<b>dependency</b>></span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p4">
<span class="s2"><b>Setup</b></span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p3">
<span class="s2"></span><br /></div>
<div class="p2">
<span class="s2">Open ports on all machines</span></div>
<div class="p2">
<span class="s2">47100 = local</span></div>
<div class="p2">
<span class="s2">48100 = shared memory</span></div>
<div class="p2">
<span class="s2"><br />
</span></div>
<div class="p2">
<span class="s2"> 90 wget http://apache.mirror.iweb.ca//ignite/1.8.0/apache-ignite-fabric-1.8.0-bin.zip</span></div>
<div class="p2">
<span class="s2"> 91 unzip apache-ignite-fabric-1.8.0-bin.zip </span></div>
<div class="p3">
<span class="s2"></span><br /></div>
<div class="p2">
<span class="s2">export IGNITE_HOME=/root/ignite-180</span></div>
<div class="p3">
<span class="s2"></span><br /></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 14.0px; font: 12.0px Courier; color: #000000; -webkit-text-stroke: #0000ee}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 14.0px; font: 12.0px Courier; color: #000000; -webkit-text-stroke: #000000}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 14.0px; font: 12.0px Courier; color: #000000; -webkit-text-stroke: #000000; min-height: 14.0px}
p.p4 {margin: 0.0px 0.0px 14.0px 0.0px; line-height: 17.0px; font: 14.0px Courier; color: #000000; -webkit-text-stroke: #000000}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 15.0px; font: 13.0px Courier; color: #000000; -webkit-text-stroke: #000000; background-color: #fcfcfc}
li.li2 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 14.0px; font: 12.0px Courier; color: #000000; -webkit-text-stroke: #000000}
span.s1 {text-decoration: underline ; font-kerning: none}
span.s2 {font-kerning: none}
span.s3 {font-kerning: none; -webkit-text-stroke: 0px #000000}
span.s4 {text-decoration: underline ; font-kerning: none; -webkit-text-stroke: 0px #0000ee}
span.s5 {font-kerning: none; -webkit-text-stroke: 0px #0000ff}
td.td1 {width: 73.6px}
td.td2 {width: 26.6px}
td.td3 {width: 56.0px}
td.td4 {width: 345.2px}
ul.ul1 {list-style-type: disc}
</style>
<br />
<div class="p5">
<span class="s5"> bin</span><span class="s2">/ignite.sh examples/config/example-ignite.xml</span></div>
<div class="p6">
<div>
<pre style="-webkit-tap-highlight-color: transparent; background: rgb(252, 252, 252); border-radius: 0px; border: 0px solid rgb(204, 204, 204); box-sizing: border-box; font-size: 13px; line-height: inherit; overflow: visible; padding: 0px 4px; position: relative; word-break: break-all; word-wrap: normal; z-index: 2;"><span style="box-sizing: border-box; padding-right: 29px;">
</span></pre>
</div>
</div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-41001740442743867262016-12-01T18:28:00.001-08:002016-12-02T10:19:37.509-08:00Running High Memory Java Applications on Amazon EC2 2048Gb 128 vCPU Memory Optimized x1.32xlarge instancesAmazon has a huge instance that we can use to run massive memory models.<br />
Here I am creating hypercube (see CM-1 from the 90's). On a normal 64G system like a Haswell 5820K board fully populated with DDR4 we top out at 22 dimensions.<br />
<br />
<br />
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ free</span></div>
<div class="p1">
<span class="s1"> total used free shared buffers cached</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852 3801596 2010941256 64 13396 238792</span></div>
<div class="p1">
<span class="s1">-/+ buffers/cache: 3549408 2011193444</span></div>
<div class="p1">
<span class="s1">Swap: 0 0 0</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -version</span></div>
<div class="p1">
<span class="s1">java version "1.7.0_121"</span></div>
<div class="p1">
<span class="s1">OpenJDK Runtime Environment (amzn-2.6.8.1.69.amzn1-x86_64 u121-b00)</span></div>
<div class="p1">
<span class="s1">OpenJDK 64-Bit Server VM (build 24.121-b00, mixed mode)</span></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ sudo yum install java-1.8.0</span></div>
<div class="p1">
<span class="s1">Loaded plugins: priorities, update-motd, upgrade-helper</span></div>
<div class="p1">
<span class="s1">amzn-main/latest | 2.1 kB 00:00 </span></div>
<div class="p1">
<span class="s1">amzn-updates/latest | 2.3 kB 00:00 </span></div>
<div class="p1">
<span class="s1">Resolving Dependencies</span></div>
<div class="p1">
<span class="s1">--> Running transaction check</span></div>
<div class="p1">
<span class="s1">---> Package java-1.8.0-openjdk.x86_64 1:1.8.0.111-1.b15.25.amzn1 will be installed</span></div>
<div class="p1">
<span class="s1">--> Processing Dependency: java-1.8.0-openjdk-headless = 1:1.8.0.111-1.b15.25.amzn1 for package: 1:java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86_64</span></div>
<div class="p1">
<span class="s1">--> Running transaction check</span></div>
<div class="p1">
<span class="s1">---> Package java-1.8.0-openjdk-headless.x86_64 1:1.8.0.111-1.b15.25.amzn1 will be installed</span></div>
<div class="p1">
<span class="s1">amzn-main/latest/filelists_db | 5.1 MB 00:00 </span></div>
<div class="p1">
<span class="s1">amzn-updates/latest/filelists_db | 1.2 MB 00:00 </span></div>
<div class="p1">
<span class="s1">--> Processing Dependency: lksctp-tools for package: 1:java-1.8.0-openjdk-headless-1.8.0.111-1.b15.25.amzn1.x86_64</span></div>
<div class="p1">
<span class="s1">--> Running transaction check</span></div>
<div class="p1">
<span class="s1">---> Package lksctp-tools.x86_64 0:1.0.10-7.7.amzn1 will be installed</span></div>
<div class="p1">
<span class="s1">--> Finished Dependency Resolution</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Dependencies Resolved</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">=======================================================================================================================================================================</span></div>
<div class="p1">
<span class="s1"> Package Arch Version Repository Size</span></div>
<div class="p1">
<span class="s1">=======================================================================================================================================================================</span></div>
<div class="p1">
<span class="s1">Installing:</span></div>
<div class="p1">
<span class="s1"> java-1.8.0-openjdk x86_64 1:1.8.0.111-1.b15.25.amzn1 amzn-updates 227 k</span></div>
<div class="p1">
<span class="s1">Installing for dependencies:</span></div>
<div class="p1">
<span class="s1"> java-1.8.0-openjdk-headless x86_64 1:1.8.0.111-1.b15.25.amzn1 amzn-updates 39 M</span></div>
<div class="p1">
<span class="s1"> lksctp-tools x86_64 1.0.10-7.7.amzn1 amzn-main 89 k</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Transaction Summary</span></div>
<div class="p1">
<span class="s1">=======================================================================================================================================================================</span></div>
<div class="p1">
<span class="s1">Install 1 Package (+2 Dependent packages)</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Total download size: 39 M</span></div>
<div class="p1">
<span class="s1">Installed size: 102 M</span></div>
<div class="p1">
<span class="s1">Is this ok [y/d/N]: y</span></div>
<div class="p1">
<span class="s1">Downloading packages:</span></div>
<div class="p1">
<span class="s1">(1/3): java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86_64.rpm | 227 kB 00:00 </span></div>
<div class="p1">
<span class="s1">(2/3): java-1.8.0-openjdk-headless-1.8.0.111-1.b15.25.amzn1.x86_64.rpm | 39 MB 00:00 </span></div>
<div class="p1">
<span class="s1">(3/3): lksctp-tools-1.0.10-7.7.amzn1.x86_64.rpm | 89 kB 00:00 </span></div>
<div class="p1">
<span class="s1">-----------------------------------------------------------------------------------------------------------------------------------------------------------------------</span></div>
<div class="p1">
<span class="s1">Total 48 MB/s | 39 MB 00:00:00 </span></div>
<div class="p1">
<span class="s1">Running transaction check</span></div>
<div class="p1">
<span class="s1">Running transaction test</span></div>
<div class="p1">
<span class="s1">Transaction test succeeded</span></div>
<div class="p1">
<span class="s1">Running transaction</span></div>
<div class="p1">
<span class="s1"> Installing : lksctp-tools-1.0.10-7.7.amzn1.x86_64 1/3 </span></div>
<div class="p1">
<span class="s1"> Installing : 1:java-1.8.0-openjdk-headless-1.8.0.111-1.b15.25.amzn1.x86_64 2/3 </span></div>
<div class="p1">
<span class="s1"> Installing : 1:java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86_64 3/3 </span></div>
<div class="p1">
<span class="s1"> Verifying : lksctp-tools-1.0.10-7.7.amzn1.x86_64 1/3 </span></div>
<div class="p1">
<span class="s1"> Verifying : 1:java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86_64 2/3 </span></div>
<div class="p1">
<span class="s1"> Verifying : 1:java-1.8.0-openjdk-headless-1.8.0.111-1.b15.25.amzn1.x86_64 3/3 </span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Installed:</span></div>
<div class="p1">
<span class="s1"> java-1.8.0-openjdk.x86_64 1:1.8.0.111-1.b15.25.amzn1 </span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Dependency Installed:</span></div>
<div class="p1">
<span class="s1"> java-1.8.0-openjdk-headless.x86_64 1:1.8.0.111-1.b15.25.amzn1 lksctp-tools.x86_64 0:1.0.10-7.7.amzn1 </span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Complete!</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ sudo alternatives --config java</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">There are 2 programs which provide 'java'.</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1"> Selection Command</span></div>
<div class="p1">
<span class="s1">-----------------------------------------------</span></div>
<div class="p1">
<span class="s1">*+ 1 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java</span></div>
<div class="p1">
<span class="s1"> 2 /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Enter to keep the current selection[+], or type selection number: 2</span></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -version</span></div>
<div class="p1">
<span class="s1">openjdk version "1.8.0_111"</span></div>
<div class="p1">
<span class="s1">OpenJDK Runtime Environment (build 1.8.0_111-b15)</span></div>
<div class="p1">
<span class="s1">OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 22 1</span></div>
<div class="p1">
<span class="s1">22 dimensional hypercube: 1480643986605</span></div>
<div class="p1">
<span class="s1">_created 46137344 bidirectional links between 4194304 routers in the 22 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">47345 time: Total Ram: 8450998272 free: 2292978440</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ free</span></div>
<div class="p1">
<span class="s1"> total used free shared buffers cached</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852 20609684 1994133168 68 14064 406412</span></div>
<div class="p1">
<span class="s1">-/+ buffers/cache: 20189208 1994553644</span></div>
<div class="p1">
<span class="s1">Swap: 0 0 0</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 22 2 -server </span></div>
<div class="p1">
<span class="s1">22 dimensional hypercube: 1480644388217</span></div>
<div class="p1">
<span class="s1">_created 46137344 bidirectional links between 4194304 routers in the 22 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">43511 time: Total Ram: 8432648192 free: 2267825616</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 23 1 </span></div>
<div class="p1">
<span class="s1">23 dimensional hypercube: 1480644473000</span></div>
<div class="p1">
<span class="s1">_created 96468992 bidirectional links between 8388608 routers in the 23 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">140562 time: Total Ram: 18184929280 free: 5885736464</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">top - 02:09:40 up 2:05, 2 users, load average: 39.26, 21.54, 10.94</span></div>
<div class="p1">
<span class="s1">Tasks: 730 total, 1 running, 729 sleeping, 0 stopped, 0 zombie</span></div>
<div class="p1">
<span class="s1">Cpu(s): 62.6%us, 0.3%sy, 0.0%ni, 37.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 19851880k used, 1994890972k free, 14168k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 406416k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<div class="p1">
<span class="s1"> 7999 ec2-user 20 0 39.9g 15g 14m S 8058.5 0.8 96:23.57 java </span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Tasks: 729 total, 2 running, 727 sleeping, 0 stopped, 0 zombie</span></div>
<div class="p1">
<span class="s1">Cpu(s): 62.8%us, 0.4%sy, 0.0%ni, 36.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 32436576k used, 1982306276k free, 14304k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 406416k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<div class="p1">
<span class="s1"> 8226 ec2-user 20 0 39.8g 27g 15m S 8098.1 1.4 255:48.30 java </span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[vagrant@nsp-latest workspace]$ free</span></div>
<div class="p1">
<span class="s1"> total used free shared buff/cache available</span></div>
<div class="p1">
<span class="s1">Mem: 7495248 263396 5030584 10720 2201268 6923444</span></div>
<div class="p1">
<span class="s1">Swap: 1048572 171480 877092</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ free</span></div>
<div class="p1">
<span class="s1"> total used free shared buffers cached</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852 33580832 1981162020 68 14372 406416</span></div>
<div class="p1">
<span class="s1">-/+ buffers/cache: 33160044 1981582808</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">Swap: 0 0 0</span><br />
<span class="s1"><br /></span>
<span class="s1"><br /></span>
<span class="s1">running with extra memory</span><br />
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -Xmx512g -Xms512g -server -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 24 1 </span></div>
<div class="p1">
<span class="s1">24 dimensional hypercube: 1480649540504</span></div>
<div class="p1">
<span class="s1">_created 201326592 bidirectional links between 16777216 routers in the 24 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">101381 time: Total Ram: 526849671168 free: 471873878864</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -Xmx512g -Xms512g -server -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 23 1 </span></div>
<div class="p1">
<span class="s1">23 dimensional hypercube: 1480649699358</span></div>
<div class="p1">
<span class="s1">_created 96468992 bidirectional links between 8388608 routers in the 23 dimensional HyperCube.</span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1">50312 time: Total Ram: 526849671168 free: 496612985560</span><br />
<span class="s1"><br /></span>
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">Cpu(s): 0.6%us, 0.2%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 52853856k used, 1961888996k free, 18704k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 436572k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1"> 43697 ec2-user 20 0 538g 46g 14m S 99.9 2.4 1:26.89 java </span></div>
<div class="p1">
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ java -Xmx512g -Xms512g -server -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 25 1 </span></div>
<div class="p1">
<span class="s1">25 dimensional hypercube: 1480649786242</span></div>
<div class="p1">
<span class="s1">_created 419430400 bidirectional links between 33554432 routers in the 25 dimensional HyperCube.</span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1">289783 time: Total Ram: 526849671168 free: 397656555744</span><br />
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ nohup java -Xmx1792g -Xms1792g -server -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 26 1 &</span></div>
<div class="p1">
<span class="s1">[1] 44231</span></div>
<div class="p1">
<span style="font-variant-ligatures: no-common-ligatures;">[ec2-user@ip-172-31-57-105 ~]$ ps -ef | grep java</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
span.s2 {font-variant-ligatures: no-common-ligatures; color: #750000}
</style>
<br />
<div class="p1">
<span class="s1">ec2-user 44231 43947 91 03:55 pts/0 00:00:05 </span><span class="s2"><b>java</b></span><span class="s1"> -Xmx1792g -Xms1792g -server -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 26 1</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ tail -f nohup.out</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
</div>
<div class="p1">
<div class="p1">
<span class="s1">2 dimensional hypercube: 1480650669281</span></div>
<div class="p1">
<span class="s1">_created 4 bidirectional links between 4 routers in the 2 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">1 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">3 dimensional hypercube: 1480650669283</span></div>
<div class="p1">
<span class="s1">_created 12 bidirectional links between 8 routers in the 3 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">0 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">4 dimensional hypercube: 1480650669283</span></div>
<div class="p1">
<span class="s1">_created 32 bidirectional links between 16 routers in the 4 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">0 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">5 dimensional hypercube: 1480650669283</span></div>
<div class="p1">
<span class="s1">_created 80 bidirectional links between 32 routers in the 5 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">1 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">6 dimensional hypercube: 1480650669284</span></div>
<div class="p1">
<span class="s1">_created 192 bidirectional links between 64 routers in the 6 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">0 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">7 dimensional hypercube: 1480650669284</span></div>
<div class="p1">
<span class="s1">_created 448 bidirectional links between 128 routers in the 7 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">1 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">8 dimensional hypercube: 1480650669286</span></div>
<div class="p1">
<span class="s1">_created 1024 bidirectional links between 256 routers in the 8 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">1 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">9 dimensional hypercube: 1480650669287</span></div>
<div class="p1">
<span class="s1">_created 2304 bidirectional links between 512 routers in the 9 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">1 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">10 dimensional hypercube: 1480650669288</span></div>
<div class="p1">
<span class="s1">_created 5120 bidirectional links between 1024 routers in the 10 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">3 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">11 dimensional hypercube: 1480650669291</span></div>
<div class="p1">
<span class="s1">_created 11264 bidirectional links between 2048 routers in the 11 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">5 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">12 dimensional hypercube: 1480650669296</span></div>
<div class="p1">
<span class="s1">_created 24576 bidirectional links between 4096 routers in the 12 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">10 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">13 dimensional hypercube: 1480650669306</span></div>
<div class="p1">
<span class="s1">_created 53248 bidirectional links between 8192 routers in the 13 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">21 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">14 dimensional hypercube: 1480650669327</span></div>
<div class="p1">
<span class="s1">_created 114688 bidirectional links between 16384 routers in the 14 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">38 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">15 dimensional hypercube: 1480650669365</span></div>
<div class="p1">
<span class="s1">_created 245760 bidirectional links between 32768 routers in the 15 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">82 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">16 dimensional hypercube: 1480650669447</span></div>
<div class="p1">
<span class="s1">_created 524288 bidirectional links between 65536 routers in the 16 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">174 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">17 dimensional hypercube: 1480650669621</span></div>
<div class="p1">
<span class="s1">_created 1114112 bidirectional links between 131072 routers in the 17 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">384 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">18 dimensional hypercube: 1480650670005</span></div>
<div class="p1">
<span class="s1">_created 2359296 bidirectional links between 262144 routers in the 18 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">923 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">19 dimensional hypercube: 1480650670928</span></div>
<div class="p1">
<span class="s1">_created 4980736 bidirectional links between 524288 routers in the 19 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">1970 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">20 dimensional hypercube: 1480650672898</span></div>
<div class="p1">
<span class="s1">_created 10485760 bidirectional links between 1048576 routers in the 20 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">4677 time: Total Ram: 1843972800512 free: 1826792931288</span></div>
<div class="p1">
<span class="s1">21 dimensional hypercube: 1480650677576</span></div>
<div class="p1">
<span class="s1">_created 22020096 bidirectional links between 2097152 routers in the 21 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">9739 time: Total Ram: 1843972800512 free: 1818202996656</span></div>
<div class="p1">
<span class="s1">22 dimensional hypercube: 1480650687314</span></div>
<div class="p1">
<span class="s1">_created 46137344 bidirectional links between 4194304 routers in the 22 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">21156 time: Total Ram: 1843972800512 free: 1809613062048</span></div>
<div class="p1">
<span class="s1">23 dimensional hypercube: 1480650708470</span></div>
<div class="p1">
<span class="s1">_created 96468992 bidirectional links between 8388608 routers in the 23 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">44953 time: Total Ram: 1843972800512 free: 1783843258176</span><br />
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">Tasks: 743 total, 1 running, 742 sleeping, 0 stopped, 0 zombie</span></div>
<div class="p1">
<span class="s1">Cpu(s): 0.6%us, 0.2%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 55841328k used, 1958901524k free, 19244k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 436796k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1"> 44386 ec2-user 20 0 1865g 49g 14m S 100.2 2.6 1:19.67 java</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
</div>
<div class="p1">
<div class="p1">
<span class="s1">24 dimensional hypercube: 1480651245506</span></div>
<div class="p1">
<span class="s1">_created 201326592 bidirectional links between 16777216 routers in the 24 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">101306 time: Total Ram: 1843972800512 free: 1732303649872</span></div>
<div class="p1">
<span class="s1">25 dimensional hypercube: 1480651346812</span></div>
<div class="p1">
<span class="s1">_created 419430400 bidirectional links between 33554432 routers in the 25 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">264376 time: Total Ram: 1843972800512 free: 1603454628832</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">Tasks: 742 total, 1 running, 741 sleeping, 0 stopped, 0 zombie</span></div>
<div class="p1">
<span class="s1">Cpu(s): 0.6%us, 0.2%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 278517036k used, 1736225816k free, 19408k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 436796k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<div class="p1">
<span class="s1"> 44386 ec2-user 20 0 1865g 261g 15m S 100.2 13.6 8:45.96 java </span></div>
<div class="p1">
<span class="s1"></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<br /></div>
<div class="p1">
<div class="p1">
<span class="s1">26 dimensional hypercube: 1480651611188</span></div>
<div class="p1">
<span class="s1">_created 872415232 bidirectional links between 67108864 routers in the 26 dimensional HyperCube.</span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1">656891 time: Total Ram: 1843972800512 free: 1647312996576</span></div>
</div>
</div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p2">
<span class="s1"></span><br />
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ free</span></div>
<div class="p1">
<span class="s1"> total used free shared buffers cached</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852 661273556 1353469296 72 19672 436804</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">-/+ buffers/cache: 660817080 1353925772</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span></div>
<div class="p1">
<span class="s1"> 44386 ec2-user 20 0 1865g 625g 15m S 99.9 32.6 82:37.39 java </span><br />
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">Cpu(s): 0.8%us, 0.0%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 1088382060k used, 926360792k free, 19984k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 436804k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1"> 44386 ec2-user 20 0 1870g 1.0t 15m S 100.2 53.7 313:16.41 java </span></div>
</div>
<div class="p1">
<span class="s1"><br /></span>
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">25 dimensional hypercube: 1480651346812</span></div>
<div class="p1">
<span class="s1">_created 419430400 bidirectional links between 33554432 routers in the 25 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">264376 time: Total Ram: 1843972800512 free: 1603454628832</span></div>
<div class="p1">
<span class="s1">26 dimensional hypercube: 1480651611188</span></div>
<div class="p1">
<span class="s1">_created 872415232 bidirectional links between 67108864 routers in the 26 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">656891 time: Total Ram: 1843972800512 free: 1647312996576</span></div>
<div class="p1">
<span class="s1">27 dimensional hypercube: 1480652268079</span></div>
<div class="p1">
<span class="s1">_created 1811939328 bidirectional links between 134217728 routers in the 27 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">1433987 time: Total Ram: 1843972800512 free: 1253174190840</span></div>
<div class="p1">
<span class="s1">28 dimensional hypercube: 1480653702066</span></div>
<div class="p1">
<span class="s1">_created 3758096384 bidirectional links between 268435456 routers in the 28 dimensional HyperCube.</span></div>
<div class="p1">
<span class="s1">4437315 time: Total Ram: 1843972800512 free: 1001914613768</span></div>
<div class="p1">
<span class="s1">29 dimensional hypercube: 1480658139381</span></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ free</span></div>
<div class="p1">
<span class="s1"> total used free shared buffers cached</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852 1992699228 22043624 64 28708 437000</span></div>
<div class="p1">
<span class="s1">-/+ buffers/cache: 1992233520 22509332</span></div>
<div class="p1">
<span class="s1">Swap: 0 0 0</span></div>
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ ps -ef | grep java</span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
span.s2 {font-variant-ligatures: no-common-ligatures; color: #750000}
</style>
</span><br />
<div class="p1">
<span class="s1">ec2-user 44386 1 99 03:59 ? 18-01:29:52 </span><span class="s2"><b>java</b></span><span class="s1"> -Xmx1792g -Xms1792g -server -cp graph-0.0.1-SNAPSHOT.jar org.obrienscience.ai.HyperCube 26 1</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">Tasks: 739 total, 1 running, 738 sleeping, 0 stopped, 0 zombie</span></div>
<div class="p1">
<span class="s1">Cpu(s): 62.6%us, 0.3%sy, 0.0%ni, 37.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 1992699740k used, 22043112k free, 28736k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 437000k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<div class="p1">
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span></div>
<div class="p1">
<span class="s1"> 44386 ec2-user 20 0 1898g 1.8t 15m S 8059.8 98.5 26052:50 java </span><br />
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">Cpu(s): 62.6%us, 0.3%sy, 0.0%ni, 37.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 1734049420k used, 280693432k free, 32136k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 437292k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1"> 45667 ec2-user 20 0 1888g 1.6t 14m S 8045.2 85.7 5836:37 java </span><br />
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">Tasks: 744 total, 1 running, 743 sleeping, 0 stopped, 0 zombie</span></div>
<div class="p1">
<span class="s1">Cpu(s): 0.8%us, 0.0%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</span></div>
<div class="p1">
<span class="s1">Mem: 2014742852k total, 1971207064k used, 43535788k free, 33936k buffers</span></div>
<div class="p1">
<span class="s1">Swap: 0k total, 0k used, 0k free, 437308k cached</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1"> 45667 ec2-user 20 0 1886g 1.8t 14m S 100.0 97.4 13311:12 java </span><br />
<span class="s1"><br /></span>
<br />
<div class="p1">
<span class="s1">[ec2-user@ip-172-31-57-105 ~]$ free</span></div>
<div class="p1">
<span class="s1"> total used free shared buffers cached</span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1">Mem: 2014742852 1971206692 43536160 64 33944 437308</span><br />
<span class="s1"><br /></span>
<div class="p1">
<span class="s1">29 dimensional hypercube: 1480681783260</span></div>
<div class="p1">
<span class="s1">_created 7784628224 bidirectional links between 536870912 routers in the 29 dimensional HyperCube.</span></div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span><br />
<div class="p1">
<span class="s1">16316219 time: Total Ram: 1843972800512 free: 147852676280</span></div>
</div>
</div>
</div>
</div>
<div class="p1">
<span class="s1"><br /></span></div>
</div>
</div>
</div>
<div class="p1">
<div class="p1">
10 min 26d 30g</div>
<div class="p1">
25 min 27d 75g</div>
<div class="p1">
75 min 28d 185g</div>
<div class="p1">
225 min 29d 460g</div>
<div class="p1">
800 min 30d 1300g</div>
</div>
</div>
</div>
</div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9; min-height: 13.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #2b66c9; background-color: #ffffff}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-29350547681028113212016-11-30T12:48:00.000-08:002017-02-16T15:18:44.999-08:00Density Functional TheoryCarbon Based Lifeforms "Only illogics can find, hidden flaws in a straight logic line. Only erratics can recognize, errors in patterns of a perfect size."<br />
<br />
Get the Thule case for your Macbook Pro 15 <a href="https://www.thule.com/en-us/ca/computer-phone-cases/laptop-cases/thule-gauntlet-30-macbook-pro-sleeve-15-_-tl_85854234856">https://www.thule.com/en-us/ca/computer-phone-cases/laptop-cases/thule-gauntlet-30-macbook-pro-sleeve-15-_-tl_85854234856</a>Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-45470348282623789262016-11-08T11:44:00.001-08:002019-07-28T07:16:27.823-07:00Neo4j 3.1 Graph Database for Distributed Development<h2>
see <a href="http://wiki.obrienlabs.cloud/display/DEV/Graph+Database+Service+with+Neo4J">http://wiki.obrienlabs.cloud/display/DEV/Graph+Database+Service+with+Neo4J</a></h2>
<h2>
<a href="http://www.neo4j.com/" style="font-size: medium; font-weight: normal;">Neo4j</a><span style="font-size: small; font-weight: normal;"> has released its 3.1.0 version of their graph database as of Dec 2016. </span></h2>
<h2>
Neo4j 3.1 Distributed Server Graph Database</h2>
This section details setting up Neo4j 3.1 Server on several metal and VM instances of RHEL 7.3<br />
follow <a href="https://neo4j.com/docs/operations-manual/current/installation/linux/tarball/">https://neo4j.com/docs/operations-manual/current/installation/linux/tarball/</a><br />
Download: neo4j-enterprise-3.1.0-unix.tar.gz<br />
copy it over to each box (I don't use a chef/ansible combo yet on these)<br />
<br />
Source<br />
<a href="https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war">https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war</a><br />
<br />
<div style="background-color: white; box-sizing: border-box; color: #333333; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 16px; margin-bottom: 16px;">
Add a node to the embedded graph db</div>
<div style="background-color: white; box-sizing: border-box; color: #333333; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 16px; margin-bottom: 16px;">
<a href="http://neo4j.ca-central-1.elasticbeanstalk.com/FrontController?action=graph">http://neo4j.ca-central-1.elasticbeanstalk.com/FrontController?action=graph</a></div>
<div style="background-color: white; box-sizing: border-box; color: #333333; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 16px; margin-bottom: 16px;">
(bolt enabled) http browser - user:neo4j pass:password</div>
<div style="background-color: white; box-sizing: border-box; color: #333333; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 16px;">
<a href="http://neo4j.ca-central-1.elasticbeanstalk.com:7575/">http://neo4j.ca-central-1.elasticbeanstalk.com:7575</a></div>
<br />
<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">[root@obrien ~]# scp michael@192.168.15.7:/Users/michael/Dropbox/neo4j/neo4j-enterprise-3.1.0-unix.tar.gz .</span></div>
[root@obrien neo4j-enterprise-3.1.0]# vi ~/.bashrc<br />
export NEO4J_HOME=/root/neo4j-enterprise-3.1.0<br />
I am blindly running as root inside my private subnet so I increase the file handle limits as<br />
<br />
vi /etc/security/limits.conf<br />
neo4j soft nofile 40000<br />
neo4j hard nofile 40000<br />
root soft nofile 40000<br />
root hard nofile 40000<br />
<div>
<br /></div>
<div>
<div>
[root@obrien bin]# ./neo4j console</div>
<div>
Starting Neo4j.</div>
<div>
2016-12-18 23:25:26.073+0000 INFO No SSL certificate found, generating a self-signed certificate..</div>
<div>
2016-12-18 23:25:26.499+0000 INFO Starting...</div>
<div>
2016-12-18 23:25:27.227+0000 INFO Bolt enabled on localhost:7687.</div>
<div>
2016-12-18 23:25:27.241+0000 INFO Initiating metrics...</div>
<div>
2016-12-18 23:25:30.054+0000 INFO Started.</div>
<div>
2016-12-18 23:25:30.239+0000 INFO Mounted REST API at: /db/manage</div>
<div>
2016-12-18 23:25:31.331+0000 INFO Remote interface available at http://localhost:7474/</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ78Qa5LBPFC5sZX3tOAgju9rCse1Ah2izjZo7mxZqr-OPwpi2uKu-BCl2l6nYrs4T0u4WmhyphenhyphenoCHTlw0QC-53F6-Lq0UfYGKdA30FmrCV8o22wxv5tyWNeeSF4MGo-D2K-vWxqTJlnESYp/s1600/Screenshot+2016-12-18+18.27.04.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="508" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ78Qa5LBPFC5sZX3tOAgju9rCse1Ah2izjZo7mxZqr-OPwpi2uKu-BCl2l6nYrs4T0u4WmhyphenhyphenoCHTlw0QC-53F6-Lq0UfYGKdA30FmrCV8o22wxv5tyWNeeSF4MGo-D2K-vWxqTJlnESYp/s640/Screenshot+2016-12-18+18.27.04.png" width="640" /></a></div>
<div>
<br /></div>
<h2>
Neo4j 3.1 Embedded Graph Database</h2>
<br />
This section details several ways to get an embedded version of Neo4j running in a spring container.<br />
<br />
<a class="external-link" href="https://neo4j.com/docs/java-reference/current/" rel="nofollow" style="background-color: white; color: #3572b0; font-family: Arial, sans-serif; font-size: 14px; text-decoration: none;">https://neo4j.com/docs/java-reference/current/</a><br />
<h3>
Results</h3>
<div>
<div>
Amazon Elastic Beanstalk deployment to t2.tiny 2.3.1</div>
<div>
<br /></div>
<div>
/biometric-nbi/FrontController?action=graph</div>
<div>
<br />
SLF4J: Found binding in [jar:file:/var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/neo4j-security-enterprise-3.1.0-M13-beta3.jar!/org/slf4j/impl/StaticLoggerBinder.class]</div>
<div>
01-Dec-2016 04:15:23.925 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.0.36</div>
<div>
01-Dec-2016 04:15:23.928 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 4.4.23-31.54.amzn1.x86_64</div>
<div>
01-Dec-2016 04:15:23.928 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.24.amzn1.x86_64/jre</div>
<div>
01-Dec-2016 04:15:23.929 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 1.8.0_101-b13</div>
<div>
01-Dec-2016 04:15:42.683 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 18422 ms</div>
<div>
register: org.obrienlabs.nbi.graph.service.HaMonitor@b37b7a9</div>
<div>
processAction: graph</div>
<div>
<state>Node[0]+0:0</state></div>
<div>
processAction: graph</div>
<div>
<state>Node[1]+1:1</state></div>
</div>
<h3>
pom.xml</h3>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <neo4j.version>3.1.0-M13-beta3</neo4j.version></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </properties></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependencies></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <groupId>org.neo4j</groupId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <artifactId>neo4j-<span class="s1">cypher</span></artifactId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <version>${neo4j.version}</version></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </dependency> </span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <groupId>org.neo4j</groupId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <artifactId>neo4j-enterprise</artifactId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <version>${neo4j.version}</version></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <groupId>org.neo4j.app</groupId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <artifactId>neo4j-server</artifactId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <version>${neo4j.version}</version></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <groupId>org.neo4j</groupId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <artifactId>neo4j-ha</artifactId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <version>${neo4j.version}</version></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </dependency></span><br />
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <groupId>org.neo4j</groupId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <artifactId>neo4j-management</artifactId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <version>${neo4j.version}</version></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </dependency></span></div>
</div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <groupId>org.neo4j.app</groupId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <artifactId>neo4j-server-enterprise</artifactId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <version>${neo4j.version}</version></span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9}
span.s1 {text-decoration: underline}
</style>
<br />
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </dependency></span><br />
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <dependency></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <groupId>org.neo4j.app</groupId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <artifactId>neo4j-bolt</artifactId></span></div>
<div class="p1">
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> <version>${neo4j.version}</version></span></div>
<br />
<span style="color: #38761d; font-family: "courier new" , "courier" , monospace;"> </dependency> </span></div>
<h3>
spring.xml (embedded)</h3>
<div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #8f9ea4}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9; min-height: 15.0px}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #f28a00}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; min-height: 15.0px}
span.s1 {color: #000000}
span.s2 {color: #e6e8e9}
span.s3 {color: #799ebf}
span.s4 {color: #f28a00}
</style>
<br />
<div class="p1">
<span style="color: #38761d;"> <context:annotation-config<span class="s1"> </span>/></span></div>
<div class="p1">
<span style="color: #38761d;"> <context:spring-configured<span class="s1"> </span>/></span></div>
<div class="p2">
<span style="color: #38761d;"><span class="s2"> </span><!-- rest annotations --></span></div>
<div class="p1">
<span style="color: #38761d;"> <mvc:annotation-driven<span class="s1"> </span>/></span></div>
<div class="p3">
<span style="color: #38761d;"> </span></div>
<div class="p2">
<span style="color: #38761d;"><span class="s2"> </span></span><span style="color: #e6e8e9; font-family: "arial"; font-size: 12px;"> </span><span style="color: #e6e8e9; font-family: "arial"; font-size: 12px;"><context:component-scan</span><span class="s1" style="font-family: "arial"; font-size: 12px;"> </span><span class="s2" style="font-family: "arial"; font-size: 12px;">base-package</span><span style="color: #e6e8e9; font-family: "arial"; font-size: 12px;">=</span><span class="s3" style="font-family: "arial"; font-size: 12px;">"org.obrienlabs.nbi.graph.service"</span><span class="s1" style="font-family: "arial"; font-size: 12px;"> </span><span style="color: #e6e8e9; font-family: "arial"; font-size: 12px;">/></span><br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #8f9ea4}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #f28a00}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; min-height: 15.0px}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9; min-height: 15.0px}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #799ebf}
span.s1 {color: #000000}
span.s2 {color: #799ebf}
span.s3 {color: #f28a00}
span.s4 {color: #e6e8e9}
</style>
<br />
<div class="p2">
<span class="s4"> </span><!-- in cases where the DAO's are in a separate jar - list them --></div>
<div class="p3">
<span class="s4"> <bean</span><span class="s1"> </span><span class="s2">id</span><span class="s4">=</span>"daoFacade"<span class="s1"> </span><span class="s2">class</span><span class="s4">=</span>"org.obrienlabs.nbi.graph.service.ApplicationService"<span class="s4">/></span></div>
<div class="p3">
<span class="s4"> <bean</span><span class="s1"> </span><span class="s2">id</span><span class="s4">=</span>"IGraphDatabaseService"<span class="s1"> </span><span class="s2">class</span><span class="s4">=</span>"org.obrienlabs.nbi.graph.service.GraphDatabaseServiceImpl"<span class="s4">/></span></div>
<div class="p4">
<br /></div>
<div class="p5">
</div>
<div class="p1">
<util:map<span class="s1"> </span><span class="s2">id</span>=<span class="s3">"config"</span>></div>
<div class="p3">
<span class="s4"> <entry</span><span class="s1"> </span><span class="s2">key</span><span class="s4">=</span>"enable_remote_shell"<span class="s1"> </span><span class="s2">value</span><span class="s4">=</span>"true"<span class="s4">/></span></div>
<div class="p1">
<entry<span class="s1"> </span><span class="s2">key</span>=<span class="s3">"ha.server_id"</span><span class="s1"> </span><span class="s2">value</span>=<span class="s3">"1"</span>/></div>
<div class="p3">
<span class="s4"> <entry</span><span class="s1"> </span><span class="s2">key</span><span class="s4">=</span>"ha.initial_hosts"<span class="s1"> </span><span class="s2">value</span><span class="s4">=</span>"127.0.0.1:5001"<span class="s4">/></span></div>
<div class="p1">
</util:map></div>
<div class="p5">
</div>
<div class="p3">
<span class="s4"> <bean</span><span class="s1"> </span><span class="s2">id</span><span class="s4">=</span>"haMonitor"<span class="s1"> </span><span class="s2">class</span><span class="s4">=</span>"com.obrienlabs.graph.service.HaMonitor"<span class="s4">/></span></div>
<div class="p3">
<span class="s4"> <bean</span><span class="s1"> </span><span class="s2">id</span><span class="s4">=</span>"graphDbFactory"<span class="s1"> </span><span class="s2">class</span><span class="s4">=</span>"com.obrienlabs.graph.service.ExtendedHighlyAvailableGraphDatabaseFactory"<span class="s4">></span></div>
<div class="p1">
<constructor-arg<span class="s1"> </span><span class="s2">ref</span>=<span class="s3">"haMonitor"</span><span class="s1"> </span>/></div>
<div class="p1">
</bean></div>
<div class="p5">
</div>
<div class="p3">
<span class="s4"> <bean</span><span class="s1"> </span><span class="s2">id</span><span class="s4">=</span>"graphDbBuilder"<span class="s1"> </span><span class="s2">factory-bean</span><span class="s4">=</span>"graphDbFactory"<span class="s1"> </span><span class="s2">factory-method</span><span class="s4">=</span>"newEmbeddedDatabaseBuilder"<span class="s4">></span></div>
<div class="p1">
<constructor-arg<span class="s1"> </span><span class="s2">value</span>=<span class="s3">"neo4j"</span>/></div>
<div class="p1">
</bean></div>
<div class="p3">
<span class="s4"> <bean</span><span class="s1"> </span><span class="s2">id</span><span class="s4">=</span>"graphDbBuilderFinal"<span class="s1"> </span><span class="s2">factory-bean</span><span class="s4">=</span>"graphDbBuilder"<span class="s1"> </span><span class="s2">factory-method</span><span class="s4">=</span>"setConfig"<span class="s4">></span></div>
<div class="p1">
<constructor-arg<span class="s1"> </span><span class="s2">ref</span>=<span class="s3">"config"</span>/></div>
<div class="p1">
</bean></div>
<div class="p3">
<span class="s4"> <bean</span><span class="s1"> </span><span class="s2">id</span><span class="s4">=</span>"graphDatabaseService"<span class="s1"> </span><span class="s2">factory-bean</span><span class="s4">=</span>"graphDbBuilderFinal"<span class="s1"> </span><span class="s2">factory-method</span><span class="s4">=</span>"newGraphDatabase"<span class="s1"> </span></div>
<div class="p6">
<span class="s1"> </span>destroy-method<span class="s4">=</span><span class="s3">"shutdown"</span><span class="s1"> </span><span class="s4">/></span></div>
</div>
</div>
<br />
<h3>
servlet</h3>
<div class="p1">
<span class="s1"> </span><b><i>@Autowired</i></b></div>
<div class="p2">
<span class="s1"> </span><span style="background-color: #0b5394;"><span class="s2"><b><i>@Qualifier</i></b></span><span class="s3">(</span>"daoFacade"<span class="s3">)</span></span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #b097c9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #f28a00}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
span.s1 {color: #e6e8e9}
span.s2 {color: #b097c9}
span.s3 {color: #ede7c4}
span.s4 {color: #a2ce76}
span.s5 {color: #8f9ea4}
</style>
<br />
<div class="p3">
<span class="s1"> </span><span class="s4"><b>private</b></span><span class="s1"> </span>ApplicationServiceLocal<span class="s1"> </span>service<span class="s3">;</span></div>
<h3>
spring bean hierarchy</h3>
<br />
<div class="p1">
<span class="s1"><b>public</b></span><span class="s2"> </span><span class="s1"><b>interface</b></span><span class="s2"> </span>ApplicationServiceLocal<span class="s2"> </span><span class="s3">{</span></div>
<div class="p2">
<span class="s2"><span class="Apple-tab-span"> </span></span><span class="s4"><b>String</b></span><span class="s2"> </span><span style="color: #0b5394;"><b>getGraph</b>();</span></div>
<div class="p2">
<span style="color: #ede7c4;">}</span></div>
<br />
<div class="p1">
<span class="s1"><b><i>@Service</i></b></span><span class="s2">(</span>"daoFacade"<span class="s2">)</span></div>
<div class="p3">
<span class="s4"><b>public</b></span><span class="s5"> </span><span class="s4"><b>class</b></span><span class="s5"> </span><b>ApplicationService</b><span class="s5"> </span><span class="s4"><b>implements</b></span><span class="s5"> </span>ApplicationServiceLocal<span class="s5"> </span><span class="s2">{</span></div>
<div class="p6">
<span class="s5"><span class="Apple-tab-span"> </span></span><b><i>@Inject</i></b></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #f28a00}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #8f9ea4}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9; min-height: 14.0px}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; min-height: 14.0px}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #b097c9}
span.s1 {color: #b097c9}
span.s2 {color: #ede7c4}
span.s3 {text-decoration: underline}
span.s4 {color: #a2ce76}
span.s5 {color: #e6e8e9}
span.s6 {color: #4abaeb}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p3">
<span class="s5"><span class="Apple-tab-span"> </span></span><span class="s4"><b>private</b></span><span class="s5"> </span>IGraphDatabaseService<span class="s5"> </span>graphDatabaseService<span class="s2">;</span></div>
<div class="p3">
<span class="s2">}</span></div>
<div class="p1">
<span class="s1"><b>public</b></span><span class="s2"> </span><span class="s1"><b>interface</b></span><span class="s2"> </span>IGraphDatabaseService<span class="s2"> </span><span class="s3">{</span><span class="Apple-tab-span" style="color: #e6e8e9;"> </span></div>
<div class="p3">
<span class="s2"><span class="Apple-tab-span"> </span></span><span class="s4">Node</span><span class="s2"> </span><b>createNode</b>();</div>
<div class="p3">
<span class="s2"><span class="Apple-tab-span"> </span></span><span class="s4">Node</span><span class="s2"> </span><b>getNodeById</b>(<span class="s2"> </span><span class="s1"><b>long</b></span><span class="s2"> id</span>);</div>
<div class="p4">
<br /></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9; min-height: 14.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #ede7c4}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; min-height: 14.0px}
span.s1 {color: #a2ce76}
span.s2 {color: #e6e8e9}
span.s3 {color: #ede7c4}
span.s4 {color: #799ebf}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p3">
}</div>
<div class="p1">
<span class="s1"><b><i>@Named</i></b></span><span class="s2">(</span>"graphDatabaseService"<span class="s2">)</span></div>
<div class="p2">
<span style="color: #0b5394;"><span class="s3"><b>public</b></span><span class="s4"> </span><span class="s3"><b>class</b></span><span class="s4"> </span><b>GraphDatabaseServiceImpl</b><span class="s4"> </span><span class="s3"><b>implements</b></span><span class="s4"> </span>IGraphDatabaseService<span class="s4"> </span><span class="s2">{</span><span class="Apple-tab-span"> </span></span></div>
<div class="p4">
<span class="s4"><span class="Apple-tab-span"> </span></span><b><i>@Inject</i></b><span class="s4"> </span></div>
<div class="p2">
<span class="s4"><span class="Apple-tab-span"> </span></span><span style="color: #0b5394;">GraphDatabaseService<span class="s4"> </span>databaseService<span class="s2">;</span></span></div>
<div class="p5">
<br /></div>
<div class="p4">
<span class="s4"><span class="Apple-tab-span"> </span></span><b><i>@Override</i></b></div>
<div class="p8">
<span class="s4"><span class="Apple-tab-span"> </span></span><span class="s3"><b>public</b></span><span class="s4"> </span><span class="s5">Node</span><span class="s4"> </span><b>createNode</b>()<span class="s4"> </span>{</div>
<div class="p7">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s5">Node</span> <b>node</b> <span class="s2">=</span> <span class="s3"><b>null</b></span><span class="s2">;</span></div>
<div class="p7">
<span class="Apple-tab-span"> </span> <span class="s3"><b>try</b></span> <span class="s2">(</span> <span style="color: #0b5394;">org<span class="s2">.</span>neo4j<span class="s2">.</span>graphdb</span><span class="s2">.</span><span class="s5">Transaction</span> <b>tx</b> <span class="s2">=</span> <span class="s5">databaseService</span><span class="s2">.</span><span style="color: #0b5394;">beginTx<span class="s2">()</span> <span class="s2">)</span> <span class="s2">{</span></span><span style="color: #0b5394;"> </span></div>
<div class="p7">
<span style="color: #0b5394;"><span class="Apple-tab-span"> </span> node <span class="s2">=</span> </span><span class="s5">databaseService</span><span class="s2">.</span><span style="color: #0b5394;">createNode<span class="s2">();</span></span></div>
<div class="p7">
<span style="color: #0b5394;"><span class="Apple-tab-span"> </span> node<span class="s2">.</span>setProperty<span class="s2">(</span> <span class="s6">"message"</span><span class="s2">,</span> <span class="s6">" node1 "</span> <span class="s2">);</span></span></div>
<div class="p7">
<span style="color: #0b5394;"><span class="Apple-tab-span"> </span> tx<span class="s2">.</span>success<span class="s2">();</span></span></div>
<div class="p7">
<span style="color: #0b5394;"><span class="Apple-tab-span"> </span> <span class="s2">}</span></span></div>
<div class="p7">
<span style="color: #0b5394;"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s3"><b>return</b></span> node<span class="s2">;</span></span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #f28a00}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9; min-height: 14.0px}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #b097c9}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; min-height: 14.0px}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #a2ce76}
p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9}
p.p8 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #ede7c4}
span.s1 {color: #b097c9}
span.s2 {color: #ede7c4}
span.s3 {color: #a2ce76}
span.s4 {color: #e6e8e9}
span.s5 {color: #799ebf}
span.s6 {color: #f28a00}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p7">
<span style="color: #0b5394;"><span class="Apple-tab-span"> </span><span class="s2">}</span></span></div>
<br />
<br />
<h3>
HA listener override</h3>
<div>
We are currently running the older paxos not the raft based causal cluster HA mode</div>
<div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">2016-12-19 00:54:47.703+0000 INFO [o.n.c.m.p.PaxosClusterMemberAvailability] Listening at:cluster://127.0.0.1:5001</span></div>
</div>
<br />
<div class="p1">
<span class="s1"><b>package</b></span><span class="s2"> org.obrienlabs.nbi.</span>graph<span class="s2">.</span>service<span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>kernel<span class="s2">.</span>ha<span class="s2">.</span>HighlyAvailableGraphDatabase<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>kernel<span class="s2">.</span>ha<span class="s2">.</span>cluster<span class="s2">.</span>HighAvailabilityMemberChangeEvent<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>kernel<span class="s2">.</span>ha<span class="s2">.</span>cluster<span class="s2">.</span>HighAvailabilityMemberListener<span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>class</b></span><span class="s3"> </span><b>HaMonitor</b><span class="s3"> </span><span class="s1"><b>implements</b></span><span class="s3"> </span>HighAvailabilityMemberListener<span class="s3"> </span><span class="s2">{</span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s3"> </span><span class="s1"><b>protected</b></span><span class="s3"> </span><b>HighlyAvailableGraphDatabase</b><span class="s3"> </span>db<span class="s2">;</span></div>
<div class="p4">
</div>
<div class="p3">
<span class="s3"> </span><span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>void</b></span><span class="s3"> </span><span class="s2"><b>setDb</b>(</span><b>HighlyAvailableGraphDatabase</b><span class="s3"> aInDb</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p1">
<span class="s4">db</span> <span class="s2">=</span> aInDb<span class="s2">;</span></div>
<div class="p1">
<span class="s2">}</span></div>
<div class="p4">
</div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span></span><b><i>@Override</i></b></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span></span><span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>void</b></span><span class="s3"> </span><span class="s2"><b>masterIsElected</b>(</span><b>HighAvailabilityMemberChangeEvent</b><span class="s3"> event</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4"><b>System</b></span><span class="s2">.</span><span class="s3"><b><i>out</i></b></span><span class="s2">.</span><span class="s4">println</span><span class="s2">(</span>"MasterIsElected"<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p2">
<br /></div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span></span><b><i>@Override</i></b></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span></span><span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>void</b></span><span class="s3"> </span><span class="s2"><b>masterIsAvailable</b>(</span><b>HighAvailabilityMemberChangeEvent</b><span class="s3"> event</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4"><b>System</b></span><span class="s2">.</span><span class="s3"><b><i>out</i></b></span><span class="s2">.</span><span class="s4">println</span><span class="s2">(</span>"masterIsAvailable"<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p2">
<br /></div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span></span><b><i>@Override</i></b></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span></span><span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>void</b></span><span class="s3"> </span><span class="s2"><b>slaveIsAvailable</b>(</span><b>HighAvailabilityMemberChangeEvent</b><span class="s3"> event</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4"><b>System</b></span><span class="s2">.</span><span class="s3"><b><i>out</i></b></span><span class="s2">.</span><span class="s4">println</span><span class="s2">(</span>"slaveIsAvailabl"<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p2">
<br /></div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span></span><b><i>@Override</i></b></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span></span><span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>void</b></span><span class="s3"> </span><span class="s2"><b>instanceStops</b>(</span><b>HighAvailabilityMemberChangeEvent</b><span class="s3"> event</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4"><b>System</b></span><span class="s2">.</span><span class="s3"><b><i>out</i></b></span><span class="s2">.</span><span class="s4">println</span><span class="s2">(</span>"instanceStops"<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p2">
<br /></div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span></span><b><i>@Override</i></b></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span></span><span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>void</b></span><span class="s3"> </span><span class="s2"><b>instanceDetached</b>(</span><b>HighAvailabilityMemberChangeEvent</b><span class="s3"> event</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4"><b>System</b></span><span class="s2">.</span><span class="s3"><b><i>out</i></b></span><span class="s2">.</span><span class="s4">println</span><span class="s2">(</span>"instanceDetached"<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; min-height: 14.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9; min-height: 14.0px}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #b097c9}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #f28a00}
p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #ede7c4}
span.s1 {color: #a2ce76}
span.s2 {color: #ede7c4}
span.s3 {color: #e6e8e9}
span.s4 {color: #799ebf}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p7">
}</div>
<br />
<div class="p1">
<span class="s1"><b>package</b></span> org<span class="s2">.</span>obrienlabs<span class="s2">.nbi.</span>graph<span class="s2">.</span>service<span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="s1"><b>import</b></span> java<span class="s2">.</span>io<span class="s2">.</span>File<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> java<span class="s2">.</span>util<span class="s2">.</span>Map<span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>graphdb<span class="s2">.</span>GraphDatabaseService<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>graphdb<span class="s2">.</span>factory<span class="s2">.</span>GraphDatabaseBuilder<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>graphdb<span class="s2">.</span>factory<span class="s2">.</span>GraphDatabaseFactoryState<span class="s2">;</span><br />
<span class="s2">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9}
span.s1 {color: #a2ce76}
span.s2 {color: #ede7c4}
</style>
</span><br />
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>graphdb<span class="s2">.</span>factory<span class="s2">.</span>GraphDatabaseSettings<span class="s2">;</span></div>
</div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>graphdb<span class="s2">.</span>factory<span class="s2">.</span>HighlyAvailableGraphDatabaseFactory<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>kernel<span class="s2">.</span>ha<span class="s2">.</span>HighlyAvailableGraphDatabase<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>kernel<span class="s2">.</span>ha<span class="s2">.</span>cluster<span class="s2">.</span>HighAvailabilityMemberStateMachine<span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>class</b></span><span class="s3"> </span><b>ExtendedHighlyAvailableGraphDatabaseFactory</b><span class="s3"> </span><span class="s1"><b>extends</b></span><span class="s3"> </span><b>HighlyAvailableGraphDatabaseFactory</b><span class="s3"> </span><span class="s2">{</span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s3"> </span><b>HaMonitor</b><span class="s3"> </span>haMonitor<span class="s2">;</span><br />
<span class="s2"> </span><span class="cm-variable" style="box-sizing: border-box; color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">GraphDatabaseSettings</span><span style="color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">.</span><span class="cm-variable" style="box-sizing: border-box; color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">BoltConnector</span><span style="color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;"> </span><span class="cm-variable" style="box-sizing: border-box; color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">bolt</span><span style="color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;"> </span><span class="cm-operator" style="box-sizing: border-box; color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">=</span><span style="color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;"> </span><span class="cm-variable" style="box-sizing: border-box; color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">GraphDatabaseSettings</span><span style="color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">.</span><span class="cm-variable" style="box-sizing: border-box; color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">boltConnector</span><span style="color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">( </span><span class="cm-string" style="box-sizing: border-box; color: #aa1111; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;">"0"</span><span style="color: inherit; font-family: "consolas" , "liberation mono" , "courier" , monospace; font-size: 0.85em; white-space: pre;"> );</span></div>
<div class="p4">
<span class="s3"> </span><span class="s1"><b>public</b></span><span class="s3"> </span><b>ExtendedHighlyAvailableGraphDatabaseFactory</b>(<span class="s4"><b>HaMonitor</b></span><span class="s3"> aInHaMonitor</span>)<span class="s3"> </span>{</div>
<div class="p1">
<span class="s4">haMonitor</span> <span class="s2">=</span> aInHaMonitor<span class="s2">;</span></div>
<div class="p1">
<span class="s2">}</span></div>
<div class="p2">
<br /></div>
<div class="p5">
<span class="s3"> </span><b><i>@Override</i></b></div>
<div class="p3">
<span class="s3"> </span><span class="s1"><b>protected</b></span><span class="s3"> </span><b>GraphDatabaseBuilder</b><span class="s2">.</span>DatabaseCreator<span class="s3"> </span><span class="s2"><b>createDatabaseCreator</b>(</span></div>
<div class="p1">
<span class="s1"><b>final</b></span> <span class="s4"><b>File</b></span> storeDir<span class="s2">,</span> <span class="s1"><b>final</b></span> <span class="s4"><b>GraphDatabaseFactoryState</b></span> state<span class="s2">)</span> <span class="s2">{</span></div>
<div class="p3">
<span class="s3"> </span><span class="s1"><b>return</b></span><span class="s3"> </span><span class="s1"><b>new</b></span><span class="s3"> </span><b>GraphDatabaseBuilder</b><span class="s2">.</span>DatabaseCreator<span class="s2">()</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="s5"><b><i>@Override</i></b></span></div>
<div class="p1">
<span class="s1"><b>public</b></span> <span class="s4">GraphDatabaseService</span> <span class="s2"><b>newDatabase</b>(</span> <span class="s1"><b>final</b></span> <span class="s4">Map</span><span class="s2"><</span>String<span class="s2">,</span> String<span class="s2">></span> config <span class="s2">)</span> <span class="s2">{</span></div>
<div class="p6">
<span class="s3"> config</span><span class="s2">.</span><span class="s3">put</span><span class="s2">(</span><span class="s3"> </span>"unsupported.dbms.ephemeral"<span class="s2">,</span><span class="s3"> </span>"false"<span class="s3"> </span><span class="s2">);</span><br />
<span class="s2" style="color: #cc0000;"> // enable bolt to enable 7474 http browser</span><br />
<span style="color: #cc0000;"><span class="s2"> </span><span class="s3">config</span><span class="s2">.</span><span class="s3">put</span><span class="s2">(</span><span class="s3"> b</span>olt.type.name()<span class="s2">,</span><span class="s3"> </span>"BOLT"<span class="s3"> </span><span class="s2">);</span></span><br />
<span class="s2" style="color: #cc0000;"><span class="s3"> config</span><span class="s2">.</span><span class="s3">put</span><span class="s2">(</span><span class="s3"> bolt.enabled.name(), </span>"true"<span class="s3"> </span><span class="s2">);</span></span><br />
<span class="s2"><span class="s2" style="color: #cc0000;"><span class="s3"> config</span><span class="s2">.</span><span class="s3">put</span><span class="s2">(</span><span class="s3"> bolt.address.name(), "localhost:7687"</span><span class="s3"> </span><span class="s2">);</span></span></span><br />
<span class="s2"> </span></div>
<div class="p3">
<span class="s3"> </span><b>HighlyAvailableGraphDatabase</b><span class="s3"> <b>haDB</b> </span><span class="s2">=</span><span class="s3"> </span></div>
<div class="p3">
<span class="s3"> <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s1"><b>new</b></span><span class="s3"> </span>HighlyAvailableGraphDatabase<span class="s2">(</span><span class="s3"> storeDir</span><span class="s2">,</span><span class="s3"> config</span><span class="s2">,</span><span class="s3"> state</span><span class="s2">.</span>databaseDependencies<span class="s2">());</span></div>
<div class="p1">
<span class="s4">haMonitor</span><span class="s2">.</span><span class="s4">setDb</span><span class="s2">(</span>haDB<span class="s2">);</span></div>
<div class="p1">
<span class="s4"><b>HighAvailabilityMemberStateMachine</b></span> <b>memberStateMachine</b> <span class="s2">=</span> </div>
<div class="p3">
<span class="s3"> <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>haDB</span><span class="s2">.</span>getDependencyResolver<span class="s2">().</span><span class="s3">resolveDependency</span><span class="s2">(</span><b>HighAvailabilityMemberStateMachine</b><span class="s2">.</span><span class="s1"><b>class</b></span><span class="s2">);</span></div>
<div class="p1">
<span class="s1"><b>if</b></span> <span class="s2">(</span> memberStateMachine <span class="s2">!=</span> <span class="s1"><b>null</b></span> <span class="s2">)</span> <span class="s2">{</span></div>
<div class="p1">
memberStateMachine<span class="s2">.</span><span class="s4">addHighAvailabilityMemberListener</span><span class="s2">(</span><span class="s4">haMonitor</span><span class="s2">);</span></div>
<div class="p1">
<span class="s4"><b>System</b></span><span class="s2">.</span><b><i>out</i></b><span class="s2">.</span><span class="s4">println</span><span class="s2">(</span><span class="s6">"register: "</span> <span class="s2">+</span> <span class="s4">haMonitor</span><span class="s2">);</span></div>
<div class="p1">
<span class="s2">}</span></div>
<div class="p1">
<span class="s1"><b>return</b></span> haDB<span class="s2">;</span></div>
<div class="p1">
<span class="s2">}</span></div>
<div class="p1">
<span class="s2">};</span></div>
<div class="p1">
<span class="s2">}</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; min-height: 14.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #ede7c4}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #b097c9}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #f28a00}
span.s1 {color: #a2ce76}
span.s2 {color: #ede7c4}
span.s3 {color: #e6e8e9}
span.s4 {color: #799ebf}
span.s5 {color: #b097c9}
span.s6 {color: #f28a00}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p4">
}</div>
<br />
<br />
Issue: constructor arg takes a file not a string anymore<br />
<pre>org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'graphDbBuilder' defined in ServletContext resource [/WEB-INF/spring.xml]: Unsatisfied dependency expressed through constructor argument with index 0 of type [java.io.File]: Could not convert factory method argument value of type [java.lang.String] to required type [java.io.File]: Failed to convert value of type [java.lang.String] to required type [java.io.File]; nested exception is java.lang.IllegalArgumentException: The resource path [/../work/graph.db] has been normalized to [null] which is not valid
</pre>
<div>
<br />
in<br />
<div class="p1">
<span class="s1"> <bean</span><span class="s2"> </span><span class="s3">id</span><span class="s1">=</span>"graphDbBuilder"<span class="s2"> </span><span class="s3">factory-bean</span><span class="s1">=</span>"graphDbFactory"<span class="s2"> </span><span class="s3">factory-method</span><span class="s1">=</span>"newEmbeddedDatabaseBuilder"<span class="s1">></span></div>
<div class="p2">
<span class="s1"> </span><!-- bean id="graphDbBuilder" factory-bean="graphDbFactory" factory-method="newHighlyAvailableDatabaseBuilder"--></div>
<div class="p3">
<constructor-arg<span class="s2"> </span><span class="s3">value</span>=<span class="s4">"neo4j"</span>/></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #f28a00}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #8f9ea4}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9}
span.s1 {color: #e6e8e9}
span.s2 {color: #000000}
span.s3 {color: #799ebf}
span.s4 {color: #f28a00}
</style>
<br />
<div class="p3">
</bean></div>
<br />
This works<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9}
span.s1 {color: #000000}
span.s2 {color: #799ebf}
span.s3 {color: #f28a00}
</style>
<br />
<div class="p1">
<constructor-arg<span class="s1"> </span><span class="s2">value</span>=<span class="s3">"neo4j"</span>/></div>
This does not anymore - we need to pass in a ${} variable via spring<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9}
span.s1 {color: #000000}
span.s2 {color: #799ebf}
span.s3 {color: #f28a00}
</style>
<br />
<div class="p1">
<constructor-arg<span class="s1"> </span><span class="s2">value</span>=<span class="s3">"../neo4j"</span>/></div>
</div>
<br />
https://neo4j.com/docs/java-reference/current/#tutorial-traversal<br />
<br />
<br />
<h3>
Getting the Neo4j 7474 http browser working on an Embedded graph.db</h3>
<div>
The 7474 browser no longer directly works out of the box on Neo4j 3.x as it did in 2.3. Partially because of the introduction of bolt.</div>
<div>
Going through the following workarounds to get it working again</div>
<div>
<br /></div>
<div>
<div style="background-color: white; box-sizing: border-box; color: #333333; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-weight: normal; line-height: 1.125; margin: 0px 150px 0px 0px; word-wrap: break-word;">
<span class="js-issue-title" style="box-sizing: border-box;"><span style="font-size: x-small;">Open a Bolt connector to your embedded instance to get GUI administration and other benefits</span></span></div>
</div>
<div>
<a href="https://github.com/neo4j/neo4j/issues/7608">https://github.com/neo4j/neo4j/issues/7608</a></div>
<div>
<a href="https://neo4j.com/developer/kb/how-do-i-enable-remote-https-access-with-neo4j-30x/">https://neo4j.com/developer/kb/how-do-i-enable-remote-https-access-with-neo4j-30x/</a></div>
<div>
section 2.14 of</div>
<div>
<a href="http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded">http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded</a></div>
<div>
<a href="https://support.neo4j.com/hc/en-us/requests/3741?page=1">https://support.neo4j.com/hc/en-us/requests/3741?page=1</a><br />
<a href="https://github.com/neo4j/neo4j/pull/6020/files">https://github.com/neo4j/neo4j/pull/6020/files</a><br />
<a href="https://github.com/neo4j/neo4j-browser/pull/229/files">https://github.com/neo4j/neo4j-browser/pull/229/files</a></div>
<br />
<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">2016-12-18 03:37:42.112+0000 INFO [o.n.k.i.DiagnosticsManager] com.sun.management.jmxremote.port = 8086</span></div>
<div class="p1">
<span class="s1">2016-12-18 03:37:42.490+0000 INFO [o.n.b.v.r.WorkerFactory] Bolt Server extension loaded.</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">2016-12-18 03:37:42.490+0000 INFO [o.n.b.v.r.WorkerFactory] Bolt enabled on localhost:7687.</span><br />
<span class="s1"><br /></span>
<span class="s1"><br /></span>
<br />
<div class="p1">
It turns out with a bit of help from the following post we can still bring up a wrapping NeoServer around our embedded mode Neo4j 3.1.0 server. This code however is not integrated yet with our HA listener overrides on the constructor tree we bootstrap in Spring - so is a POC. </div>
<div class="p2">
<br /></div>
<div class="p1">
With this NeoServer wrapper code - we don’t need the secondary workaround where we bring up a 2nd Neo4j server mode instance, bring up its’ 7474 browser and modify the bolt address (default 7687) in the config to point to the exposed (7688) embedded mode graph.db - without bringing either Neo4j server down as long as they point to different graph.db instances - in effect sharing the 7474 browser as a bolt client.</div>
<div class="p2">
<br /></div>
<div class="p1">
Disclaimer: I believe a proper architecture involves a clustered set of server mode bolt enabled Neo4j servers.</div>
<div class="p1">
This undocumented/internal API is used as a temporary workaround to get the neo4j browser up on an embedded Neo4j server in 3.1.0 as it used to work under the WrappingNeoServerBootstrapper in 2.3 so we can continue to use in-jvm Dijkstra algos until we are able to rework/benchmark the same or greater traversal speed via a bolt enabled cluster.</div>
<div class="p2">
<br /></div>
<div class="p1">
FrontController/spring service bean starts graph</div>
<div class="p1">
```</div>
<div class="p1">
2016-12-20 01:28:04.186+0000 INFO Starting...</div>
<div class="p1">
2016-12-20 01:28:04.533+0000 INFO Bolt enabled on localhost:7687.</div>
<div class="p1">
2016-12-20 01:28:04.534+0000 INFO Initiating metrics...</div>
<div class="p1">
2016-12-20 01:28:06.091+0000 INFO Started.</div>
<div class="p1">
2016-12-20 01:28:07.009+0000 INFO Remote interface available at http://localhost:7575/</div>
<div class="p1">
<state>community single [/ec2-user/graph2.db] : community single [/ec2-user/graph2.db/data/databases/graph.db]</state></div>
<div class="p1">
```</div>
<div class="p2">
<br /></div>
<div class="p1">
after opening browser</div>
<div class="p2">
<br /></div>
<div class="p1">
```</div>
<div class="p1">
<span class="Apple-tab-span"> </span>// http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper</div>
<div class="p1">
<span class="Apple-tab-span"> </span>// Warning: internal/unsupported API - NeoServer wrappers are usually only run in server mode - not embedded</div>
<div class="p1">
ServerBootstrapper serverBootstrapper = new EnterpriseBootstrapper();//CommunityBootstrapper();</div>
<div class="p1">
NeoServer directNeoServer = serverBootstrapper.getServer();</div>
<div class="p1">
int i = serverBootstrapper.start(new File("/ec2-user"), // will resolve to /ec2-user/data/databases/graph.db </div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Optional.of(new File("/Users/michaeal/Documents/Neo4j/.neo4j.conf")), // stub only</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>Pair.of("dbms.connector.http.address","0.0.0.0:7575"),</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>Pair.of("dbms.connector.http.enabled","true" ),</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>Pair.of("dbms.connector.http.type", "HTTP" ),</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>Pair.of("dbms.connector.http.tls_level", "DISABLED")</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>// BOLT not advised if we run an embedded Neo4j DB inside a NeoServer wrapper</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>// BOLT should be enabled if you run a standalone server's browser against this embedded bolt port</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>// WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use. WebSocket `readyState` is: 3</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>,Pair.of("dbms.connector.bolt.address","0.0.0.0:7688"),</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>Pair.of("dbms.connector.bolt.enabled","true" ),</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>Pair.of("dbms.connector.bolt.type", "HTTP" ),</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span>Pair.of("dbms.connector.bolt.tls_level", "DISABLED") <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>);</div>
<div class="p1">
NeoServer neoServer = serverBootstrapper.getServer();</div>
<div class="p1">
GraphDatabaseService graph = neoServer.getDatabase().getGraph(); </div>
<div class="p1">
```</div>
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
Debugging we see that we get the proper Jetty server available usually only to server mode Neo4j not embedded mode.</div>
<div class="p1">
```</div>
<div class="p1">
AbstractNeoServer.<clinit>() line: 114<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90<span class="Apple-tab-span"> </span></div>
<div class="p1">
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
CommunityNeoServer.<clinit>() line: 54<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90<span class="Apple-tab-span"> </span></div>
<div class="p1">
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
Config.get(Setting<T>) line: 137<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(AbstractNeoServer).<init>(Config, Database$Factory, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 159<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(CommunityNeoServer).<init>(Config, Database$Factory, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 69<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer.<init>(Config, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 105<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90<span class="Apple-tab-span"> </span></div>
<div class="p1">
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
Jetty9WebServer.<init>(LogProvider, Config) line: 135<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(CommunityNeoServer).createWebServer() line: 89<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer.createWebServer() line: 131<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(AbstractNeoServer).init() line: 181<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(AbstractNeoServer).start() line: 196<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91<span class="Apple-tab-span"> </span></div>
<div class="p1">
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
GraphDatabaseFacadeFactory.initFacade(File, Map<String,String>, Dependencies, GraphDatabaseFacade) line: 141<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseGraphDatabase.<init>(File, Map<String,String>, Dependencies) line: 36<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer.lambda$static$1(Config, GraphDatabaseFacadeFactory$Dependencies) line: 90<span class="Apple-tab-span"> </span></div>
<div class="p1">
1274309106.newGraphDatabase(Config, GraphDatabaseFacadeFactory$Dependencies) line: not available<span class="Apple-tab-span"> </span></div>
<div class="p1">
LifecycleManagingDatabase.start() line: 89<span class="Apple-tab-span"> </span></div>
<div class="p1">
LifeSupport$LifecycleInstance.start() line: 433<span class="Apple-tab-span"> </span></div>
<div class="p1">
LifeSupport.start() line: 107<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(AbstractNeoServer).start() line: 199<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91<span class="Apple-tab-span"> </span></div>
<div class="p1">
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
Jetty9WebServer.loadJAXRSResource(SessionManager, String, JaxRsServletHolderFactory) line: 536<span class="Apple-tab-span"> </span></div>
<div class="p1">
Jetty9WebServer.loadJAXRSClasses(SessionManager, String) line: 522<span class="Apple-tab-span"> </span></div>
<div class="p1">
Jetty9WebServer.loadAllMounts() line: 409<span class="Apple-tab-span"> </span></div>
<div class="p1">
Jetty9WebServer.start() line: 171<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(AbstractNeoServer).startWebServer() line: 326<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseNeoServer(AbstractNeoServer).start() line: 218<span class="Apple-tab-span"> </span></div>
<div class="p1">
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91<span class="Apple-tab-span"> </span></div>
<div class="p1">
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
serverBootstrapper<span class="Apple-tab-span"> </span>EnterpriseBootstrapper (id=299)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>dependencies<span class="Apple-tab-span"> </span>GraphDatabaseDependencies (id=347)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>log<span class="Apple-tab-span"> </span>FormattedLog (id=573)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>server<span class="Apple-tab-span"> </span>EnterpriseNeoServer (id=357)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>serverAddress<span class="Apple-tab-span"> </span>"0.0.0.0:7575" (id=575)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>shutdownHook<span class="Apple-tab-span"> </span>ServerBootstrapper$1 (id=577)<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
neoServer<span class="Apple-tab-span"> </span>EnterpriseNeoServer (id=357)<span class="Apple-tab-span"> </span></div>
<div class="p1">
….<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>webServer<span class="Apple-tab-span"> </span>Jetty9WebServer (id=406)<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
from NeoServer</div>
<div class="p1">
graph<span class="Apple-tab-span"> </span>EnterpriseGraphDatabase (id=417)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>contextFactory<span class="Apple-tab-span"> </span>Neo4jTransactionalContextFactory (id=1119)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>defaultTransactionTimeout<span class="Apple-tab-span"> </span>0<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>indexManager<span class="Apple-tab-span"> </span>Suppliers$1 (id=1121)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>nodeActions<span class="Apple-tab-span"> </span>StandardNodeActions (id=1123)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>relActions<span class="Apple-tab-span"> </span>StandardRelationshipActions (id=1128)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>schema<span class="Apple-tab-span"> </span>SchemaImpl (id=1132)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>spi<span class="Apple-tab-span"> </span>ClassicCoreSPI (id=425)<span class="Apple-tab-span"> </span></div>
<div class="p2">
<br /></div>
<div class="p1">
Without NeoServer</div>
<div class="p1">
graphDb<span class="Apple-tab-span"> </span>GraphDatabaseFacade (id=150)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>contextFactory<span class="Apple-tab-span"> </span>Neo4jTransactionalContextFactory (id=1182)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>defaultTransactionTimeout<span class="Apple-tab-span"> </span>0<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>indexManager<span class="Apple-tab-span"> </span>Suppliers$1 (id=1183)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>nodeActions<span class="Apple-tab-span"> </span>StandardNodeActions (id=1184)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>relActions<span class="Apple-tab-span"> </span>StandardRelationshipActions (id=1208)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>schema<span class="Apple-tab-span"> </span>SchemaImpl (id=1219)<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span>spi<span class="Apple-tab-span"> </span>ClassicCoreSPI (id=160)<span class="Apple-tab-span"> </span></div>
<div class="p1">
```</div>
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
```</div>
<div class="p1">
2016-12-20 02:15:02.825+0000 INFO Starting...</div>
<div class="p1">
2016-12-20 02:15:03.304+0000 INFO Bolt enabled on 0.0.0.0:7688.</div>
<div class="p1">
2016-12-20 02:15:03.316+0000 INFO Initiating metrics...</div>
<div class="p1">
2016-12-20 02:15:06.232+0000 INFO Started.</div>
<div class="p1">
2016-12-20 02:15:06.412+0000 INFO Mounted REST API at: /db/manage</div>
<div class="p1">
2016-12-20 02:15:07.103+0000 INFO Remote interface available at http://localhost:7575/</div>
<div class="p1">
<state> : enterprise single [/ec2-user/data/databases/graph.db]</state></div>
<div class="p1">
```</div>
<div class="p2">
<br /></div>
<div class="p1">
settings and logs</div>
<div class="p1">
```</div>
<div class="p1">
cat /ec2-user/logs/debug.log</div>
<div class="p1">
2016-12-20 02:15:03.225+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.enabled=true</div>
<div class="p1">
2016-12-20 02:15:03.225+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.security.auth_enabled=true</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.type=HTTP</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.directories.neo4j_home=/ec2-user</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.directories.import=import</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.tls_level=DISABLED</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.type=HTTP</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.logs.http.enabled=false</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.edition=enterprise</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.address=0.0.0.0:7688</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.address=0.0.0.0:7575</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.enabled=true</div>
<div class="p1">
2016-12-20 02:15:03.226+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.tls_level=DISABLED</div>
<div class="p1">
2016-12-20 02:15:03.304+0000 INFO [o.n.b.v.r.WorkerFactory] Bolt Server extension loaded.</div>
<div class="p1">
2016-12-20 02:15:03.304+0000 INFO [o.n.b.v.r.WorkerFactory] Bolt enabled on 0.0.0.0:7688.</div>
<div class="p1">
2016-12-20 02:15:04.687+0000 INFO [o.n.k.i.DiagnosticsManager] Kernel version: 3.1.0,16a782b42d76ca37db72958eb2565cf6aa671a29</div>
<div class="p1">
```</div>
<div class="p2">
<br /></div>
<div class="p1">
These errors are non-fatal related to JMX and can be fixed</div>
<div class="p1">
```</div>
<div class="p1">
2016-12-20 02:17:30.759+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method writeBufSize, reference 59659a03-461e-4532-9b98-7afe3a2e5704. Exception invoking method writeBufSize</div>
<div class="p1">
javax.management.RuntimeOperationsException: Exception invoking method writeBufSize</div>
<div class="p1">
<span class="Apple-tab-span"> </span>at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(BaseModelMBean.java:196)</div>
<div class="p1">
<span class="Apple-tab-span"> </span>at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)</div>
<div class="p1">
<span class="Apple-tab-span"> </span>at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)</div>
<div class="p1">
<span class="Apple-tab-span"> </span>at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:131)</div>
<div class="p1">
Caused by: java.lang.NullPointerException</div>
<div class="p1">
<span class="Apple-tab-span"> </span>at org.apache.tomcat.util.net.SocketProperties.getTxBufSize(SocketProperties.java:300)</div>
<div class="p1">
```</div>
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
7575 http and 7688 bolt ports</div>
<div class="p1">
```</div>
<div class="p1">
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7575</div>
<div class="p1">
tcp46 0 0 *.7575 *.* LISTEN 131072 131072 49013 0</div>
<div class="p1">
tcp4 0 0 127.0.0.1.7575 127.0.0.1.60685 TIME_WAIT 407296 146988 49013 0</div>
<div class="p1">
tcp6 0 0 ::1.7575 ::1.60699 TIME_WAIT 407284 146808 49013 0</div>
<div class="p1">
tcp6 0 0 ::1.7575 ::1.60700 TIME_WAIT 407284 146808 49013 0</div>
<div class="p1">
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7688</div>
<div class="p1">
tcp6 0 0 ::1.7688 ::1.60704 ESTABLISHED 406582 146808 49013 0</div>
<div class="p1">
tcp6 0 0 ::1.60704 ::1.7688 ESTABLISHED 398196 146808 48165 0</div>
<div class="p1">
tcp6 0 0 ::1.7688 ::1.60702 ESTABLISHED 406570 146808 49013 0</div>
<div class="p1">
tcp6 0 0 ::1.60702 ::1.7688 ESTABLISHED 398185 146808 48165 0</div>
<div class="p1">
tcp6 0 0 ::1.7688 ::1.60701 ESTABLISHED 407255 146808 49013 0</div>
<div class="p1">
tcp6 0 0 ::1.60701 ::1.7688 ESTABLISHED 407628 146808 48165 0</div>
<div class="p1">
tcp46 0 0 *.7688 *.* LISTEN 131072 131072 49013 0</div>
<div class="p1">
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080</div>
<div class="p1">
tcp4 0 0 127.0.0.1.8080 127.0.0.1.60584 FIN_WAIT_2 408104 146988 49013 0</div>
<div class="p1">
tcp4 994 0 127.0.0.1.60584 127.0.0.1.8080 CLOSE_WAIT 408128 146988 42992 0</div>
<div class="p1">
tcp46 0 0 *.8080 *.* LISTEN 131072 131072 49013 0</div>
<div class="p1">
udp4 0 0 *.* *.* 196724 9216 38080 0</div>
<div class="p1">
```</div>
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
This code however is not integrated yet with our HA listener overrides on the constructor tree we bootstrap in Spring - the following scrubbed/distilled mirror POC of our corporate production code will be adjusted in the next post.</div>
<div class="p2">
<br /></div>
<div class="p1">
```</div>
<div class="p1">
public class ExtendedHighlyAvailableGraphDatabaseFactory extends HighlyAvailableGraphDatabaseFactory {</div>
<div class="p1">
private HaMonitor haMonitor;</div>
<div class="p1">
private GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );</div>
<div class="p1">
@Override</div>
<div class="p1">
protected GraphDatabaseBuilder.DatabaseCreator createDatabaseCreator(</div>
<div class="p1">
final File storeDir, final GraphDatabaseFactoryState state) {</div>
<div class="p1">
return new GraphDatabaseBuilder.DatabaseCreator() {</div>
<div class="p1">
@Override</div>
<div class="p1">
public GraphDatabaseService newDatabase( final Map<String, String> config ) {</div>
<div class="p1">
HighlyAvailableGraphDatabase haDB = </div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>new HighlyAvailableGraphDatabase( storeDir, config, state.databaseDependencies());</div>
<div class="p1">
haMonitor.setDb(haDB);</div>
<div class="p1">
HighAvailabilityMemberStateMachine memberStateMachine = </div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>haDB.getDependencyResolver().resolveDependency(HighAvailabilityMemberStateMachine.class);</div>
<div class="p1">
if ( memberStateMachine != null ) {</div>
<div class="p1">
memberStateMachine.addHighAvailabilityMemberListener(haMonitor);</div>
<div class="p1">
}</div>
<div class="p1">
return haDB;</div>
<div class="p1">
} };}</div>
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #000000}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #000000; min-height: 14.0px}
span.Apple-tab-span {white-space:pre}
</style>
</span><br />
<div class="p1">
```<br />
<h4>
Neo4j 3.1.0 HA (paxos) Embedded mode HighlyAvailableGraphDatabase wrapped with an EnterpriseBootstrapper NeoServer</h4>
<div>
see</div>
<div>
<a href="http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper/41271855#41271855">http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper/41271855#41271855</a></div>
<div>
<br /></div>
In this code we get everything (except causal cluster using the raft protocol).<br />
<br />
With a few modifications I was able to wrap a HighlyAvailableGraphDatabase with a EnterpriseBootstrapper.<br />
<br />
There are some non-fatal exceptions around JMX reporting in debug.log likely related to my 8.0.28 tomcat version but the graph db running embedded in Tomcat is OK.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:00.574+0000 INFO Bolt enabled on 0.0.0.0:7688.</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:09.554+0000 INFO Attempting to join cluster of [127.0.0.1:5001]</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:11.566+0000 INFO Creating new cluster with name [neo4j.ha]...</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:11.607+0000 INFO Instance 1 (this server) entered the cluster</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:12.164+0000 INFO I am 1, moving to master</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:12.293+0000 INFO Instance 1 (this server) was elected as coordinator</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:12.462+0000 INFO I am 1, successfully moved to master</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:12.513+0000 INFO Instance 1 (this server) is available as master at ha://127.0.0.1:6001?serverId=1 with StoreId{creationTime=1482199697648, randomId=7800059877674392627, storeVersion=15531981201765894, upgradeTime=1482199697648, upgradeId=1}</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:14.495+0000 INFO Database available for write transactions</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:31.917+0000 INFO Mounted REST API at: /db/manage</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 16:20:53.264+0000 INFO Remote interface available at http://localhost:7575/</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> register: org.obrienlabs.nbi.graph.service.HaMonitor@1c0f80c9</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> public class ExtendedHighlyAvailableGraphDatabaseFactory extends HighlyAvailableGraphDatabaseFactory {</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> private HaMonitor haMonitor; </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> public GraphDatabaseService getEnterpriseGraphDatabase(final File _storeDir, final GraphDatabaseFactoryState _state, </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>final Map<String, String> _config) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>String graphPath = "/ec2-user/graph2.db" ;</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>// http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>// Warning: internal/unsupported API - NeoServer wrappers are usually only run in server mode - not embedded</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> EnterpriseBootstrapper serverBootstrapper = new EnterpriseBootstrapper();//CommunityBootstrapper();</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> int i = serverBootstrapper.start(new File("/ec2-user"), // will resolve to /ec2-user/data/databases/graph.db </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>Optional.of(new File("/some-path/Neo4j/.neo4j.conf")), // stub only</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Pair.of("dbms.connector.http.address","0.0.0.0:7575"),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("dbms.connector.http.enabled","true" ),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("dbms.connector.http.type", "HTTP" ),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("dbms.connector.http.tls_level", "DISABLED"),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Pair.of("dbms.connector.bolt.address","0.0.0.0:7688"),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("dbms.connector.bolt.enabled","true" ),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("dbms.connector.bolt.type", "HTTP" ),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("dbms.connector.bolt.tls_level", "DISABLED") ,</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("ha.initial_hosts", "127.0.0.1:5001"),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> Pair.of("ha.server_id", "1"),</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>Pair.of("dbms.mode", EnterpriseNeoServer.Mode.HA.toString()));</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> NeoServer neoServer = serverBootstrapper.getServer();</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> GraphDatabaseService graph = neoServer.getDatabase().getGraph(); </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> return graph;</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> @Override</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> protected GraphDatabaseBuilder.DatabaseCreator createDatabaseCreator(</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> final File storeDir, final GraphDatabaseFactoryState state) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> return new GraphDatabaseBuilder.DatabaseCreator() {</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> @Override</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> public GraphDatabaseService newDatabase( final Map<String, String> config ) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // section 2.14 of http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> /*HighlyAvailableGraphDatabase haDB = </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>new HighlyAvailableGraphDatabase( storeDir, config, state.databaseDependencies());*/</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> HighlyAvailableGraphDatabase haDB =(HighlyAvailableGraphDatabase) getEnterpriseGraphDatabase( storeDir, state, config);</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> haMonitor.setDb(haDB);</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // Add the HA monitor listener</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> HighAvailabilityMemberStateMachine memberStateMachine = </span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>haDB.getDependencyResolver().resolveDependency(HighAvailabilityMemberStateMachine.class);</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> if ( memberStateMachine != null ) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> memberStateMachine.addHighAvailabilityMemberListener(haMonitor);</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> return haDB;</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> }};}</span><br />
<br />
Transactions go through<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:51:07.478+0000 INFO [o.n.k.i.s.c.CountsTracker] About to rotate counts store at transaction 9 to [/ec2-user/data/databases/graph.db/neostore.counts.db.b], from [/ec2-user/data/databases/graph.db/neostore.counts.db.a].</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:51:07.480+0000 INFO [o.n.k.i.s.c.CountsTracker] Successfully rotated counts store at transaction 9 to [/ec2-user/data/databases/graph.db/neostore.counts.db.b], from [/ec2-user/data/databases/graph.db/neostore.counts.db.a].</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:51:07.483+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [9]: Store flush completed</span><br />
<br />
Settings from the mbean in jconsole are<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.503+0000 INFO [o.n.k.i.DiagnosticsManager] --- REQUESTED diagnostics START ---</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.503+0000 INFO [o.n.k.i.DiagnosticsManager] Graph Database: enterprise ha StoreId{creationTime=1482199697648, randomId=7800059877674392627, storeVersion=15531981201765894, upgradeTime=1482199697648, upgradeId=1}</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.503+0000 INFO [o.n.k.i.DiagnosticsManager] Kernel version: 3.1.0,16a782b42d76ca37db72958eb2565cf6aa671a29</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.503+0000 INFO [o.n.k.i.DiagnosticsManager] Neo4j Kernel properties:</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.enabled=true</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.mode=HA</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.block_size.labels=56</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.security.auth_enabled=true</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.type=HTTP</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.directories.neo4j_home=/ec2-user</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.directories.import=import</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.ephemeral=false</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.tls_level=DISABLED</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.type=HTTP</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] ha.server_id=1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.block_size.strings=120</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.logs.http.enabled=false</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.block_size.array_properties=120</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] ha.initial_hosts=127.0.0.1:5001</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.allow_format_migration=false</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.edition=enterprise</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.address=0.0.0.0:7688</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.address=0.0.0.0:7575</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.enabled=true</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> 2016-12-21 20:55:46.504+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.tls_level=DISABLED</span><br />
<br />
7575(jetty server) and 7688 (bolt) ports are open<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7575</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp46 0 0 *.7575 *.* LISTEN 131072 131072 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp4 0 0 127.0.0.1.7575 127.0.0.1.60685 TIME_WAIT 407296 146988 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.7575 ::1.60699 TIME_WAIT 407284 146808 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.7575 ::1.60700 TIME_WAIT 407284 146808 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7688</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.7688 ::1.60704 ESTABLISHED 406582 146808 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.60704 ::1.7688 ESTABLISHED 398196 146808 48165 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.7688 ::1.60702 ESTABLISHED 406570 146808 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.60702 ::1.7688 ESTABLISHED 398185 146808 48165 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.7688 ::1.60701 ESTABLISHED 407255 146808 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp6 0 0 ::1.60701 ::1.7688 ESTABLISHED 407628 146808 48165 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp46 0 0 *.7688 *.* LISTEN 131072 131072 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp4 0 0 127.0.0.1.8080 127.0.0.1.60584 FIN_WAIT_2 408104 146988 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp4 994 0 127.0.0.1.60584 127.0.0.1.8080 CLOSE_WAIT 408128 146988 42992 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> tcp46 0 0 *.8080 *.* LISTEN 131072 131072 49013 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> udp4 0 0 *.* *.* 196724 9216 38080 0</span><br />
<br />
Disclaimer: I believe a proper architecture involves a clustered set of server mode bolt enabled Neo4j servers using causal clustering under the raft protocol.<br />
This undocumented/internal API is used as a temporary workaround to get the neo4j browser up on an embedded Neo4j server in 3.1.0 as it used to work under the WrappingNeoServerBootstrapper in 2.3. Use of the embedded server for faster traversals was the use case - performance must be reevaluated to see if a bolt based architecture is more optimal for the traversal API.<br />
/michael</div>
</div>
<h3>
Neo4j 3.1 Technology</h3>
https://neo4j.com/blog/neo4j-3-1-beta-release/?ref=home<br />
- causal clustering (with read replicas and synced read/write clusters)<br />
- bolt can drive load balancing (with read/write splitting)<br />
- 4 security roles with list and terminate control<br />
- node and id reallocation from deleted records (uuid reuse?)<br />
- max query duration<br />
<br />
<h3>
Test Code</h3>
Run the following to bring up a fully populated hypercube and test the memory limits of your server and browser<br />
<div class="p1">
<span class="s1"><b>package</b></span> org<span class="s2">.</span>obrienlabs<span class="s2">.</span>nbi<span class="s2">.</span>graph<span class="s2">.</span>test<span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="s1"><b>import</b></span> <span class="s1"><b>static</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>driver<span class="s2">.</span>v1<span class="s2">.</span>Values<span class="s2">.</span><i>parameters</i><span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>driver<span class="s2">.</span>v1<span class="s2">.</span>AuthTokens<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>driver<span class="s2">.</span>v1<span class="s2">.</span>Driver<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>driver<span class="s2">.</span>v1<span class="s2">.</span>GraphDatabase<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>driver<span class="s2">.</span>v1<span class="s2">.</span>Session<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>driver<span class="s2">.</span>v1<span class="s2">.</span>StatementResult<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> org<span class="s2">.</span>neo4j<span class="s2">.</span>driver<span class="s2">.</span>v1<span class="s2">.</span>Transaction<span class="s2">;</span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>class</b></span><span class="s3"> </span><b>BoltUnitTest</b><span class="s3"> </span><span class="s2">{</span></div>
<div class="p2">
<br /></div>
<div class="p4">
<span class="s3"><span class="Apple-tab-span"> </span></span><b>public</b><span class="s3"> </span><b>static</b><span class="s3"> </span><b>long</b><span class="s3"> </span><span class="s4"><i>linkStep</i></span><span class="s3"> </span><span class="s2">=</span><span class="s3"> </span><span class="s5">0</span><span class="s2">;</span></div>
<div class="p4">
<span class="s3"><span class="Apple-tab-span"> </span></span><b>public</b><span class="s3"> </span><b>static</b><span class="s3"> </span><b>long</b><span class="s3"> </span><span class="s4"><i>linkCount</i></span><span class="s3"> </span><span class="s2">=</span><span class="s3"> </span><span class="s5">0</span><span class="s2">;</span></div>
<div class="p4">
<span class="s3"><span class="Apple-tab-span"> </span></span><b>public</b><span class="s3"> </span><b>static</b><span class="s3"> </span><b>long</b><span class="s3"> </span><span class="s4"><i>pk</i></span><span class="s3"> </span><span class="s2">=</span><span class="s3"> </span><span class="s5">0</span><span class="s2">;</span></div>
<div class="p4">
<span class="s3"><span class="Apple-tab-span"> </span></span><b>public</b><span class="s3"> </span><b>static</b><span class="s3"> </span><b>long</b><span class="s3"> </span><span class="s4"><i>pair</i></span><span class="s3"> </span><span class="s2">=</span><span class="s3"> </span><span class="s5">0</span><span class="s2">;</span></div>
<div class="p4">
<span class="s3"><span class="Apple-tab-span"> </span></span><b>public</b><span class="s3"> </span><b>static</b><span class="s3"> </span><b>long</b><span class="s3"> </span><span class="s4"><i>linkMax</i></span><span class="s3"> </span><span class="s2">=</span><span class="s3"> </span><span class="s5">0</span><span class="s2">;</span></div>
<div class="p5">
<span class="s3"> </span><span class="s1"><b>private</b></span><span class="s3"> </span><span class="s1"><b>static</b></span><span class="s3"> </span><span class="s1"><b>final</b></span><span class="s3"> </span><span class="s4"><b>String</b></span><span class="s3"> <b><i>url</i></b> </span><span class="s2">=</span><span class="s3"> </span>"bolt://192.168.0.26:7687"<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>private</b></span> <span class="s1"><b>static</b></span> <span class="s1"><b>final</b></span> <span class="s4"><b>String</b></span> <b><i>username</i></b> <span class="s2">=</span> <span class="s7">"neo4j"</span><span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>private</b></span> <span class="s1"><b>static</b></span> <span class="s1"><b>final</b></span> <span class="s4"><b>String</b></span> <b><i>password</i></b> <span class="s2">=</span> <span class="s7">"mainstreet"</span><span class="s2">;</span> </div>
<div class="p7">
<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s1"><b>private</b></span> <span class="s1"><b>void</b></span> <span class="s2"><b>link</b>(</span><span class="s4"><b>String</b></span> label<span class="s2">,</span> <span class="s1"><b>long</b></span> pair0<span class="s2">,</span> <span class="s1"><b>long</b></span> pair1<span class="s2">,</span> <span class="s1"><b>long</b></span> pk0<span class="s2">,</span> <span class="s1"><b>long</b></span> pk1<span class="s2">,</span> <span class="s4">Session</span> session<span class="s2">)</span> <span class="s2">{</span></div>
<div class="p1">
<span class="s1"><b>try</b></span> <span class="s2">(</span><span class="s4">Transaction</span> <b>writeTransaction</b> <span class="s2">=</span> session<span class="s2">.</span>beginTransaction<span class="s2">())</span> <span class="s2">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>writeTransaction<span class="s2">.</span>success<span class="s2">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>writeTransaction<span class="s2">.</span>run<span class="s2">(</span> </div>
<div class="p6">
<span class="s3"> <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>"MATCH (a:Node"<span class="s2">+</span><span class="s3"> pair0 </span><span class="s2">+</span><span class="s3"> </span>"),(b:Node"<span class="s2">+</span><span class="s3"> pair1 </span><span class="s2">+</span><span class="s3"> </span>") WHERE a.name = {p0} AND b.name = {p1} CREATE (a)-[r:"<span class="s3"> </span><span class="s2">+</span><span class="s3"> label </span><span class="s2">+</span><span class="s3"> </span>"]->(b)"<span class="s2">,</span></div>
<div class="p1">
<i>parameters</i><span class="s2">(</span> <span class="s7">"p0"</span><span class="s2">,</span> pk0<span class="s2">,</span> <span class="s7">"p1"</span><span class="s2">,</span> pk1<span class="s2">));</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">StatementResult</span> <b>edge1</b> <span class="s2">=</span> writeTransaction<span class="s2">.</span>run<span class="s2">(</span> </div>
<div class="p6">
<span class="s3"> <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>"MATCH (a:Node"<span class="s2">+</span><span class="s3"> pair0 </span><span class="s2">+</span><span class="s3"> </span>"),(b:Node"<span class="s2">+</span><span class="s3"> pair1 </span><span class="s2">+</span><span class="s3"> </span>") WHERE a.name = {p0} AND b.name = {p1} CREATE (a)-[r:"<span class="s3"> </span><span class="s2">+</span><span class="s3"> label </span><span class="s2">+</span><span class="s3"> </span>"]->(b) RETURN r"<span class="s2">,</span></div>
<div class="p1">
<i>parameters</i><span class="s2">(</span> <span class="s7">"p0"</span><span class="s2">,</span> pk1<span class="s2">,</span> <span class="s7">"p1"</span><span class="s2">,</span> pk0<span class="s2">));</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>writeTransaction<span class="s2">.</span>success<span class="s2">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4"><i>linkStep</i></span><span class="s2">--;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4"><i>linkCount</i></span><span class="s2">++;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1"><b>if</b></span><span class="s2">(</span><span class="s4"><i>linkStep</i></span> <span class="s2"><</span> <span class="s5">0</span><span class="s2">)</span> <span class="s2">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4"><i>linkStep</i></span> <span class="s2">=</span> <span class="s5">16</span><span class="s2">;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4"><b>System</b></span><span class="s2">.</span><b><i>out</i></b><span class="s2">.</span><span class="s4">println</span><span class="s2">(</span><span class="s7">"link: "</span> <span class="s2">+</span> pk0 <span class="s2">+</span> <span class="s7">"-"</span> <span class="s2">+</span> pk1 <span class="s2">+</span> <span class="s7">" : "</span> <span class="s2">+</span> <span class="s4"><i>linkCount</i></span> <span class="s2">+</span> <span class="s7">" of "</span> <span class="s2">+</span> <span class="s4"><i>linkMax</i></span><span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p1">
<span class="s2">}</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s1"><b>private</b></span> <span class="s1"><b>void</b></span> <span class="s2"><b>create</b>(</span><span class="s1"><b>long</b></span> pk0<span class="s2">,</span> <span class="s1"><b>long</b></span> pair<span class="s2">,</span> <span class="s4">Session</span> session<span class="s2">)</span> <span class="s2">{</span></div>
<div class="p1">
<span class="s1"><b>try</b></span> <span class="s2">(</span><span class="s4">Transaction</span> <b>writeTransaction</b> <span class="s2">=</span> session<span class="s2">.</span>beginTransaction<span class="s2">())</span> <span class="s2">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">StatementResult</span> <b>left</b> <span class="s2">=</span> writeTransaction<span class="s2">.</span>run<span class="s2">(</span> <span class="s7">"CREATE (a:Node"</span> <span class="s2">+</span> pair <span class="s2">+</span> <span class="s7">" {name: {pk}}) return a"</span><span class="s2">,</span></div>
<div class="p1">
<i>parameters</i><span class="s2">(</span> <span class="s7">"pk"</span><span class="s2">,</span> <span class="s4"><i>pk</i></span><span class="s2">));</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4"><i>pk</i></span><span class="s2">++;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>writeTransaction<span class="s2">.</span>success<span class="s2">();</span></div>
<div class="p1">
<span class="s2">}</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s1"><b>private</b></span> <span class="s1"><b>void</b></span> <span class="s2"><b>createL</b>(</span><span class="s1"><b>int</b></span> power<span class="s2">,</span> <span class="s1"><b>long</b></span> id<span class="s2">,</span> <span class="s4">Session</span> session<span class="s2">)</span> <span class="s2">{</span><span class="Apple-tab-span"> </span></div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>// base</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1"><b>if</b></span><span class="s2">(</span>power <span class="s2">==</span> <span class="s5">0</span><span class="s2">)</span> <span class="s2">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s4">create</span><span class="s2">(</span>id<span class="s2">,</span> <span class="s5">0</span><span class="s2">,</span> session<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s4">create</span><span class="s2">(</span>id <span class="s2">+</span> <span class="s5">1</span><span class="s2">,</span> <span class="s5">0</span><span class="s2">,</span> session<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s4">link</span><span class="s2">(</span><span class="s7">"L0"</span><span class="s2">,</span> <span class="s5">0</span><span class="s2">,</span> <span class="s5">0</span><span class="s2">,</span> id <span class="s2">,</span>id <span class="s2">+</span> <span class="s5">1</span><span class="s2">,</span> session<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s8">//pair++;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s1"><b>return</b></span><span class="s2">;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p7">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>// recursive</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1"><b>int</b></span> <b>ext</b> <span class="s2">=</span> <span class="s5">1</span> <span class="s2"><<</span> power<span class="s2">;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">createL</span><span class="s2">(</span>power <span class="s2">-</span> <span class="s5">1</span><span class="s2">,</span> id<span class="s2">,</span> session<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">createL</span><span class="s2">(</span>power <span class="s2">-</span> <span class="s5">1</span><span class="s2">,</span> id <span class="s2">+</span> ext<span class="s2">,</span> session<span class="s2">);</span></div>
<div class="p7">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1"><b>for</b></span><span class="s2">(</span><span class="s1"><b>int</b></span> <b>i</b><span class="s2">=</span><span class="s5">0</span><span class="s2">;</span> i<span class="s2"><</span>ext<span class="s2">;</span> i<span class="s2">++)</span> <span class="s2">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">link</span><span class="s2">(</span><span class="s7">"L"</span> <span class="s2">+</span> power<span class="s2">,</span> <span class="s5">0</span><span class="s2">,</span> <span class="s5">0</span><span class="s2">,</span> id <span class="s2">+</span> i<span class="s2">,</span> id <span class="s2">+</span> ext <span class="s2">+</span> i<span class="s2">,</span> session<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p7">
<span class="Apple-tab-span"> </span></div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span></span>// match(n:Node0) where n.name=0 return(n);</div>
<div class="p8">
<span class="s3"><span class="Apple-tab-span"> </span> </span><span class="s1"><b>public</b></span><span class="s3"> </span><span class="s1"><b>void</b></span><span class="s3"> </span><b>populateHypercube</b>()<span class="s3"> </span><span class="s1"><b>throws</b></span><span class="s3"> </span><span class="s4"><b>Exception</b></span><span class="s3"> </span>{</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="s1"><b>long</b></span> <b>count</b> <span class="s2">=</span> <span class="s5">0</span><span class="s2">;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="s1"><b>int</b></span> <b>lastPower</b> <span class="s2">=</span> <span class="s5">16</span><span class="s2">;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="s1"><b>int</b></span> <b>power</b> <span class="s2">=</span> <span class="s5">16</span><span class="s2">;</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="s4"><i>linkMax</i></span> <span class="s2">=</span> <span class="s2">(</span><span class="s5">1</span> <span class="s2"><<</span> power<span class="s2">)</span> <span class="s2">*</span> <span class="s2">(</span>power <span class="s2">+</span> <span class="s5">1</span><span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s4">Driver</span> <b>driver</b> <span class="s2">=</span> <span class="s4"><b>GraphDatabase</b></span><span class="s2">.</span><i>driver</i><span class="s2">(</span><b><i>url</i></b><span class="s2">,</span> <span class="s4"><b>AuthTokens</b></span><span class="s2">.</span><i>basic</i><span class="s2">(</span><b><i>username</i></b><span class="s2">,</span> <b><i>password</i></b><span class="s2">));</span> </div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s1"><b>try</b></span> <span class="s2">(</span><span class="s4">Session</span> <b>session</b> <span class="s2">=</span> driver<span class="s2">.</span>session<span class="s2">())</span> <span class="s2">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s1"><b>try</b></span> <span class="s2">(</span><span class="s4">Transaction</span> <b>writeTransaction</b> <span class="s2">=</span> session<span class="s2">.</span>beginTransaction<span class="s2">())</span> <span class="s2">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span><span class="s1"><b>for</b></span><span class="s2">(</span><span class="s1"><b>int</b></span> <b>i</b><span class="s2">=</span><span class="s5">0</span><span class="s2">;</span>i<span class="s2"><</span>lastPower <span class="s2">+</span> <span class="s5">1</span><span class="s2">;</span>i<span class="s2">++)</span> <span class="s2">{</span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>writeTransaction</span><span class="s2">.</span><span class="s3">run</span><span class="s2">(</span>" match (:Node0)-[r:L"<span class="s3"> </span><span class="s2">+</span><span class="s3"> i </span><span class="s2">+</span><span class="s3"> </span>"]->(:Node0) delete r"<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> writeTransaction<span class="s2">.</span>run<span class="s2">(</span><span class="s7">" match(n) delete(n)"</span><span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> writeTransaction<span class="s2">.</span>success<span class="s2">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s2">}</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="Apple-tab-span"> </span><span class="s4">createL</span><span class="s2">(</span>power<span class="s2">,</span> count<span class="s2">,</span> session<span class="s2">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s2">}</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> driver<span class="s2">.</span>close<span class="s2">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span> <span class="s2">}</span></div>
<div class="p7">
<span class="Apple-tab-span"> </span> </div>
<div class="p5">
<span class="s3"><span class="Apple-tab-span"> </span></span>// https://github.com/neo4j/neo4j-java-driver/blob/1.1/examples/src/main/java/org/neo4j/docs/driver/MinimalWorkingExample.java</div>
<div class="p7">
<span class="Apple-tab-span"> </span></div>
<div class="p4">
<span class="s3"><span class="Apple-tab-span"> </span></span><b>public</b><span class="s3"> </span><b>static</b><span class="s3"> </span><b>void</b><span class="s3"> </span><span class="s2"><b>main</b>(</span><span class="s4"><b>String</b></span><span class="s2">[]</span><span class="s3"> args</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><b>BoltUnitTest</b><span class="s3"> <b>test</b> </span><span class="s2">=</span><span class="s3"> </span><span class="s1"><b>new</b></span><span class="s3"> </span>BoltUnitTest<span class="s2">();</span></div>
<div class="p4">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><b>try</b><span class="s3"> </span><span class="s2">{</span></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>test</span><span class="s2">.</span>populateHypercube<span class="s2">();</span></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2">}</span><span class="s3"> </span><span class="s1"><b>catch</b></span><span class="s2">(</span><b>Exception</b><span class="s3"> <b>e</b></span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>e</span><span class="s2">.</span>printStackTrace<span class="s2">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; min-height: 14.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #a2ce76}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #8f9ea4}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #f28a00}
p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9; min-height: 14.0px}
p.p8 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #ede7c4}
span.s1 {color: #a2ce76}
span.s2 {color: #ede7c4}
span.s3 {color: #e6e8e9}
span.s4 {color: #799ebf}
span.s5 {color: #ffd42b}
span.s6 {text-decoration: underline}
span.s7 {color: #f28a00}
span.s8 {color: #8f9ea4}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p8">
}</div>
<h3>
Troubleshooting</h3>
./neo4j start does not work<br />
Fix: make sure your java 8 home is set like<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home</span></div>
<br />
Links<br />
<br />
<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
Workaround code wraps a Neo4j 3,1 embedded HA graph DB with EnterpriseBootstrapper to get the http browser via bolt.<a href="https://t.co/Nv1bHWPj5p">https://t.co/Nv1bHWPj5p</a> <a href="https://t.co/ludKSZ1IwM">pic.twitter.com/ludKSZ1IwM</a></div>
— Michael_OBrien (@_mikeobrien) <a href="https://twitter.com/_mikeobrien/status/813262328892420096">December 26, 2016</a></blockquote>
<script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script>
<br />
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="https://github.com/neo4j/neo4j/wiki/Neo4j-3.0-changelog" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://github.com/neo4j/neo4j/wiki/Neo4j-3.0-changelog</a></span></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="https://github.com/neo4j/neo4j/wiki/Neo4j-3.1-changelog" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://github.com/neo4j/neo4j/wiki/Neo4j-3.1-changelog</a></span></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<a class="external-link" href="https://neo4j.com/docs/operations-manual/current/deployment/upgrade/" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://neo4j.com/docs/operations-manual/current/deployment/upgrade/</a></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="https://neo4j.com/docs/java-reference/current/" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://neo4j.com/docs/java-reference/current/</a></span></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="https://neo4j.com/docs/operations-manual/current/introduction/" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://neo4j.com/docs/operations-manual/current/introduction/</a></span></div>
<a class="external-link" href="https://neo4j.com/docs/java-reference/current/javadocs/deprecated-list.html" rel="nofollow" style="background-color: white; color: #3572b0; font-family: Arial, sans-serif; font-size: 14px; text-decoration: none;">https://neo4j.com/docs/java-reference/current/javadocs/deprecated-list.html</a><br />
<div class="p2" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="https://neo4j.com/docs/operations-manual/current/deployment/high-availability/" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://neo4j.com/docs/operations-manual/current/deployment/high-availability/</a></span></div>
<div class="p2" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper" rel="nofollow" style="color: #3572b0; text-decoration: none;">http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper</a></span></div>
<div class="p2" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1">references deprecated wrapper</span></div>
<div class="p2" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="http://docs.spring.io/autorepo/docs/spring-data-neo4j/3.0.0.M1/reference/html/setup.html" rel="nofollow" style="color: #3572b0; text-decoration: none;">http://docs.spring.io/autorepo/docs/spring-data-neo4j/3.0.0.M1/reference/html/setup.html</a></span></div>
<div class="p2" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded" rel="nofollow" style="color: #3572b0; text-decoration: none;">http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded</a></span></div>
<div class="p2" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="https://github.com/neo4j/neo4j/commit/a0463303e460c7b551af7ba15d4add9c2c2d34e2" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://github.com/neo4j/neo4j/commit/a0463303e460c7b551af7ba15d4add9c2c2d34e2</a></span></div>
<div class="p2" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin-top: 10px; padding: 0px;">
<span class="s1"><a class="external-link" href="https://repo1.maven.org/maven2/org/neo4j/neo4j-deferred-locks/2.3.7/" rel="nofollow" style="color: #3572b0; text-decoration: none;">https://repo1.maven.org/maven2/org/neo4j/neo4j-deferred-locks/2.3.7/</a></span></div>
<br />
<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #799ebf}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #e6e8e9; min-height: 14.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #8f9ea4}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #ede7c4}
span.s1 {color: #a2ce76}
span.s2 {color: #e6e8e9}
span.s3 {color: #ede7c4}
span.s4 {color: #799ebf}
span.Apple-tab-span {white-space:pre}
</style><br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #8f9ea4}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #f28a00}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9; min-height: 15.0px}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #e6e8e9}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #799ebf}
span.s1 {color: #e6e8e9}
span.s2 {color: #000000}
span.s3 {color: #799ebf}
span.s4 {color: #f28a00}
</style>Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com1tag:blogger.com,1999:blog-3810253236812062548.post-69299296430976024862016-10-01T17:05:00.000-07:002016-10-02T19:59:18.857-07:00Machine Learning using Apache Spark on top of Apache Hadoop and KafkaHadoop processes historical data where Spark processes real time iOT data as it comes in for trends.<br />
<br />
http://www.cs.toronto.edu/~hinton/science.pdf<br />
<br />
MapReduce Batch Processing<br />
<a href="http://hadoop.apache.org/">http://hadoop.apache.org/</a><br />
<a href="http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3-src.tar.gz">http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3-src.tar.gz</a><br />
<a href="http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz">http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz</a><br />
<a href="http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html">http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html</a><br />
<a href="https://hadoop.apache.org/docs/r2.7.2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html">https://hadoop.apache.org/docs/r2.7.2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html</a><br />
<br />
Realtime Streaming Processing<br />
runs on Hadoop<br />
<a href="http://spark.apache.org/">http://spark.apache.org/</a><br />
<br />
Distributed Streaming Platform<br />
runs on Scala on top of Zookeeper<br />
<a href="http://kafka.apache.org/">http://kafka.apache.org/</a><br />
<a href="https://kafka.apache.org/08/quickstart.html">https://kafka.apache.org/08/quickstart.html</a><br />
<br />
<br />
output<br />
<div class="p1">
<span class="s1">[root@nuc12-i7 hadoop-2.7.3]# vi /home/hadoop-2.7.3/etc/hadoop/hadoop-env.sh</span></div>
<div class="p1">
<span class="s1">[root@nuc12-i7 hadoop-2.7.3]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep input output 'dfs[a-z.]+'</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:47 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:47 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO input.FileInputFormat: Total input paths to process : 8</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapreduce.JobSubmitter: number of splits:8</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1447651192_0001</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapreduce.Job: The url to track the job: http://localhost:8080/</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapreduce.Job: Running job: job_local1447651192_0001</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: OutputCommitter set in config null</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Waiting for map tasks</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/hadoop-policy.xml:0+9683</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Spilling map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufend = 17; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Finished spill 0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000000_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000000_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000001_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/kms-site.xml:0+5511</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000001_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000001_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000001_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000002_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/capacity-scheduler.xml:0+4436</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000002_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000002_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000002_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000003_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/kms-acls.xml:0+3518</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000003_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000003_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000003_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000004_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/hdfs-site.xml:0+775</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000004_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000004_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000004_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000005_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/core-site.xml:0+774</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000005_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000005_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000005_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000006_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/yarn-site.xml:0+690</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000006_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000006_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000006_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_m_000007_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/input/httpfs-site.xml:0+620</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_m_000007_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_m_000007_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_m_000007_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: map task executor complete.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Waiting for reduce tasks</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1447651192_0001_r_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@619f906a</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=334338464, maxSingleShuffleLimit=83584616, mergeThreshold=220663392, ioSortFactor=10, memToMemMergeOutputsThreshold=10</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.EventFetcher: attempt_local1447651192_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000003_0 decomp: 2 len: 6 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1447651192_0001_m_000003_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->2</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000000_0 decomp: 21 len: 25 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 21 bytes from map-output for attempt_local1447651192_0001_m_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 21, inMemoryMapOutputs.size() -> 2, commitMemory -> 2, usedMemory ->23</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000006_0 decomp: 2 len: 6 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1447651192_0001_m_000006_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 3, commitMemory -> 23, usedMemory ->25</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000007_0 decomp: 2 len: 6 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1447651192_0001_m_000007_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 4, commitMemory -> 25, usedMemory ->27</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000004_0 decomp: 2 len: 6 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1447651192_0001_m_000004_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 5, commitMemory -> 27, usedMemory ->29</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000001_0 decomp: 2 len: 6 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1447651192_0001_m_000001_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 6, commitMemory -> 29, usedMemory ->31</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000005_0 decomp: 2 len: 6 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1447651192_0001_m_000005_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 7, commitMemory -> 31, usedMemory ->33</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1447651192_0001_m_000002_0 decomp: 2 len: 6 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1447651192_0001_m_000002_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 8, commitMemory -> 33, usedMemory ->35</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: 8 / 8 copied.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: finalMerge called with 8 in-memory map-outputs and 0 on-disk map-outputs</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Merger: Merging 8 sorted segments</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 10 bytes</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: Merged 8 segments, 35 bytes to disk to satisfy reduce memory limit</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: Merging 1 files, 25 bytes from disk</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Merger: Merging 1 sorted segments</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 10 bytes</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: 8 / 8 copied.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task:attempt_local1447651192_0001_r_000000_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: 8 / 8 copied.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task attempt_local1447651192_0001_r_000000_0 is allowed to commit now</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1447651192_0001_r_000000_0' to file:/home/hadoop-2.7.3/grep-temp-965988443/_temporary/0/task_local1447651192_0001_r_000000</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: reduce > reduce</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.Task: Task 'attempt_local1447651192_0001_r_000000_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: Finishing task: attempt_local1447651192_0001_r_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:48 INFO mapred.LocalJobRunner: reduce task executor complete.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.Job: Job job_local1447651192_0001 running in uber mode : false</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.Job: map 100% reduce 100%</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.Job: Job job_local1447651192_0001 completed successfully</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.Job: Counters: 30</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>File System Counters</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of bytes read=2892701</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of bytes written=5231540</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of read operations=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of large read operations=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of write operations=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>Map-Reduce Framework</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map input records=745</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map output records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map output bytes=17</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map output materialized bytes=67</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Input split bytes=877</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Combine input records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Combine output records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce input groups=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce shuffle bytes=67</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce input records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce output records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Spilled Records=2</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Shuffled Maps =8</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Failed Shuffles=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Merged Map outputs=8</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>GC time elapsed (ms)=90</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Total committed heap usage (bytes)=3163029504</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>Shuffle Errors</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>BAD_ID=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>CONNECTION=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>IO_ERROR=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>WRONG_LENGTH=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>WRONG_MAP=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>WRONG_REDUCE=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>File Input Format Counters </span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Bytes Read=26007</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>File Output Format Counters </span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Bytes Written=123</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO input.FileInputFormat: Total input paths to process : 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.JobSubmitter: number of splits:1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1445128842_0002</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.Job: The url to track the job: http://localhost:8080/</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: OutputCommitter set in config null</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapreduce.Job: Running job: job_local1445128842_0002</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: Waiting for map tasks</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: Starting task: attempt_local1445128842_0002_m_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: Processing split: file:/home/hadoop-2.7.3/grep-temp-965988443/part-r-00000:0+111</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: soft limit at 83886080</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: </span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: Starting flush of map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: Spilling map output</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: bufstart = 0; bufend = 17; bufvoid = 104857600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.MapTask: Finished spill 0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Task: Task:attempt_local1445128842_0002_m_000000_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: map</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Task: Task 'attempt_local1445128842_0002_m_000000_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: Finishing task: attempt_local1445128842_0002_m_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: map task executor complete.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: Waiting for reduce tasks</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: Starting task: attempt_local1445128842_0002_r_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1673725c</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=334338464, maxSingleShuffleLimit=83584616, mergeThreshold=220663392, ioSortFactor=10, memToMemMergeOutputsThreshold=10</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.EventFetcher: attempt_local1445128842_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.LocalFetcher: localfetcher#2 about to shuffle output of map attempt_local1445128842_0002_m_000000_0 decomp: 21 len: 25 to MEMORY</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.InMemoryMapOutput: Read 21 bytes from map-output for attempt_local1445128842_0002_m_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 21, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->21</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 WARN io.ReadaheadPool: Failed readahead on ifile</span></div>
<div class="p1">
<span class="s1">EBADF: Bad file descriptor</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>at java.lang.Thread.run(Thread.java:745)</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: 1 / 1 copied.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Merger: Merging 1 sorted segments</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 11 bytes</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.MergeManagerImpl: Merged 1 segments, 21 bytes to disk to satisfy reduce memory limit</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.MergeManagerImpl: Merging 1 files, 25 bytes from disk</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Merger: Merging 1 sorted segments</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 11 bytes</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: 1 / 1 copied.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Task: Task:attempt_local1445128842_0002_r_000000_0 is done. And is in the process of committing</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: 1 / 1 copied.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Task: Task attempt_local1445128842_0002_r_000000_0 is allowed to commit now</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1445128842_0002_r_000000_0' to file:/home/hadoop-2.7.3/output/_temporary/0/task_local1445128842_0002_r_000000</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: reduce > reduce</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.Task: Task 'attempt_local1445128842_0002_r_000000_0' done.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: Finishing task: attempt_local1445128842_0002_r_000000_0</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:49 INFO mapred.LocalJobRunner: reduce task executor complete.</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:50 INFO mapreduce.Job: Job job_local1445128842_0002 running in uber mode : false</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:50 INFO mapreduce.Job: map 100% reduce 100%</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:50 INFO mapreduce.Job: Job job_local1445128842_0002 completed successfully</span></div>
<div class="p1">
<span class="s1">16/10/02 00:16:50 INFO mapreduce.Job: Counters: 30</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>File System Counters</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of bytes read=1248270</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of bytes written=2321184</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of read operations=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of large read operations=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>FILE: Number of write operations=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>Map-Reduce Framework</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map input records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map output records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map output bytes=17</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Map output materialized bytes=25</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Input split bytes=121</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Combine input records=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Combine output records=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce input groups=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce shuffle bytes=25</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce input records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Reduce output records=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Spilled Records=2</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Shuffled Maps =1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Failed Shuffles=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Merged Map outputs=1</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>GC time elapsed (ms)=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Total committed heap usage (bytes)=695205888</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>Shuffle Errors</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>BAD_ID=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>CONNECTION=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>IO_ERROR=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>WRONG_LENGTH=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>WRONG_MAP=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>WRONG_REDUCE=0</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>File Input Format Counters </span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Bytes Read=123</span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span>File Output Format Counters </span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Bytes Written=23</span></div>
<div class="p1">
<span class="s1">[root@nuc12-i7 hadoop-2.7.3]# cat output/*</span></div>
<div class="p1">
<span class="s1">1<span class="Apple-tab-span"> </span>dfsadmin</span></div>
<div class="p1">
<span class="s1">[root@nuc12-i7 hadoop-2.7.3]# ls output</span></div>
<br />
<div class="p1">
<span class="s1">part-r-00000 _SUCCESS</span></div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-90496890237168248222016-06-07T08:20:00.002-07:002016-06-07T18:05:31.706-07:00JAX-RS 2.0 Client API on Tomcat 8 running Spring 4, Hibernate 4 with a Jersey 2 RestController<h3>
Rest Client</h3>
<div>
<div class="p1">
<span class="s1"><b>package</b></span> com<span class="s2">.</span>obrienlabs<span class="s2">.</span>gps<span class="s2">.</span>integration<span class="s2">;</span></div>
<div class="p2">
</div>
<div class="p1">
<span class="s1"><b>import</b></span> javax<span class="s2">.</span>ws<span class="s2">.</span>rs<span class="s2">.</span>client<span class="s2">.</span>Client<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> javax<span class="s2">.</span>ws<span class="s2">.</span>rs<span class="s2">.</span>client<span class="s2">.</span>ClientBuilder<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> javax<span class="s2">.</span>ws<span class="s2">.</span>rs<span class="s2">.</span>client<span class="s2">.</span>Entity<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> javax<span class="s2">.</span>ws<span class="s2">.</span>rs<span class="s2">.</span>client<span class="s2">.</span>WebTarget<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> javax<span class="s2">.</span>ws<span class="s2">.</span>rs<span class="s2">.</span>core<span class="s2">.</span>Response<span class="s2">;</span></div>
<div class="p2">
</div>
<div class="p1">
<span class="s1"><b>import</b></span> com<span class="s2">.</span>obrienlabs<span class="s2">.</span>gps<span class="s2">.</span>business<span class="s2">.</span>entity<span class="s2">.</span>Record<span class="s2">;</span></div>
<div class="p1">
<span class="s1"><b>import</b></span> com<span class="s2">.</span>obrienlabs<span class="s2">.</span>gps<span class="s2">.</span>business<span class="s2">.</span>entity<span class="s2">.</span>User<span class="s2">;</span></div>
<div class="p2">
</div>
<div class="p3">
<b>public</b><span class="s3"> </span><b>class</b><span class="s3"> </span><span class="s4"><b>JAXRSClient</b></span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p2">
</div>
<div class="p3">
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span></span><b>public</b><span class="s1"> </span><b>void</b><span class="s1"> </span><span class="s2"><b>run</b>()</span><span class="s1"> </span><span class="s2">{</span></div>
<div class="p2">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s3"><b>Long</b></span> <b>id</b> <span class="s2">=</span> <span class="s4">1L</span><span class="s2">;</span></div>
<div class="p2">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s3">Client</span> <b>client</b> <span class="s2">=</span> <span class="s5"><b>ClientBuilder</b></span><span class="s2">.</span><i>newClient</i><span class="s2">();</span></div>
<div class="p3">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s3">WebTarget</span><span class="s1"> <b>rootTarget</b> </span><span class="s2">=</span><span class="s1"> client</span><span class="s2">.</span><span class="s1">target</span><span class="s2">(</span>"http://127.0.0.1:8080/biometric/rest/read"<span class="s2">);</span></div>
<div class="p2">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s3">WebTarget</span> <b>latestTarget</b> <span class="s2">=</span> rootTarget<span class="s2">.</span>path<span class="s2">(</span><span class="s6">"json/latest"</span><span class="s2">);</span></div>
<div class="p2">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s3">WebTarget</span> <b>finalTarget</b> <span class="s2">=</span> latestTarget<span class="s2">.</span>path<span class="s2">(</span>id<span class="s2">.</span><span class="s3">toString</span><span class="s2">());</span></div>
<div class="p2">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s3"><b>Record</b></span> <b>record</b> <span class="s2">=</span> finalTarget<span class="s2">.</span>request<span class="s2">().</span>get<span class="s2">(</span><span class="s3"><b>Record</b></span><span class="s2">.</span><span class="s7"><b>class</b></span><span class="s2">);</span></div>
<div class="p5">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><b>System</b><span class="s2">.</span><span class="s1"><b><i>out</i></b></span><span class="s2">.</span>println<span class="s2">(</span><span class="s1">record</span><span class="s2">);</span></div>
<div class="p2">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>client<span class="s2">.</span>close<span class="s2">();</span></div>
<br />
<div class="p2">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
</div>
<div class="p7">
<span class="Apple-tab-span"> </span></div>
<div class="p7">
<span class="Apple-tab-span"> </span></div>
<div class="p3">
<span class="s3"> </span><b>public</b><span class="s3"> </span><b>static</b><span class="s3"> </span><b>void</b><span class="s3"> </span><span class="s2"><b>main</b>(</span><span class="s4"><b>String</b></span><span class="s2">[]</span><span class="s3"> args</span><span class="s2">)</span><span class="s3"> </span><span class="s2">{</span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><b>JAXRSClient</b><span class="s3"> <b>client</b> </span><span class="s2">=</span><span class="s3"> </span><span class="s1"><b>new</b></span><span class="s3"> </span>JAXRSClient<span class="s2">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>client<span class="s2">.</span><span class="s4">run</span><span class="s2">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="s2">}</span></div>
</div>
<div>
</div>
<h3>
Rest Backend </h3>
<div>
<br /></div>
<div>
<h4>
pom.xml</h4>
</div>
<div>
<br /></div>
<h4>
RestController</h4>
<div>
<div class="p1">
<span class="s1"><b><i>@Path</i></b></span><span class="s2">(</span>"/read"<span class="s2">)</span></div>
<div class="p2">
////@Produces("text/plain")</div>
<div class="p2">
//@Consumes("application/<span class="s3">json</span>")</div>
<div class="p2">
//@RestController</div>
<div class="p2">
//@Service("daoFacade")</div>
<div class="p2">
//@TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = true)</div>
<div class="p3">
<b><i>@RequestMapping</i></b><span class="s2">(</span><span class="s4">"/read"</span><span class="s2">)</span></div>
<div class="p3">
<b><i>@Component</i></b></div>
<div class="p4">
<span class="s5"><b>public</b></span><span class="s6"> </span><span class="s5"><b>class</b></span><span class="s6"> </span><b>RestServiceImpl</b><span class="s6"> </span><span class="s5"><b>extends</b></span><span class="s6"> </span><b>Application</b><span class="s6"> </span><span class="s2">{</span></div>
<div class="p2">
<span class="s6"><span class="Apple-tab-span"> </span></span>// https://jersey.java.net/documentation/latest/spring.html</div>
<div class="p2">
<span class="s6"><span class="Apple-tab-span"> </span></span>// http://stackoverflow.com/questions/3027834/inject-a-ejb-into-jax-rs-restfull-service</div>
<div class="p2">
<span class="s6"><span class="Apple-tab-span"> </span></span>// not supported until 2.0 https://java.net/jira/browse/JERSEY-517</div>
<div class="p2">
<span class="s6"><span class="Apple-tab-span"> </span></span>//@EJB(name="<span class="s3">ejb</span>/ApplicationService") // not for JAX-RS</div>
<div class="p2">
<span class="s6"><span class="Apple-tab-span"> </span></span>// 20160606: spring injection requires jersey-spring3 from org.glassfish.jersey.ext with spring 4 exclusions in the <span class="s3">pom</span></div>
<div class="p3">
<span class="s6"><span class="Apple-tab-span"> </span></span><b><i>@Inject</i></b></div>
<div class="p2">
<span class="s6"><span class="Apple-tab-span"> </span></span>//@Autowired</div>
<div class="p1">
<span class="s6"><span class="Apple-tab-span"> </span></span><span class="s1"><b><i>@Qualifier</i></b></span><span class="s2">(</span>"daoFacade"<span class="s2">)</span></div>
<br />
<div class="p4">
<span class="s6"> </span><span class="s5"><b>private</b></span><span class="s6"> </span>ApplicationServiceLocal<span class="s6"> </span>applicationServiceLocal<span class="s2">;</span><span class="s7">// = new ApplicationService();<span class="Apple-tab-span"> </span></span></div>
<div class="p4">
<span class="s7"><br /></span></div>
<div class="p1">
<span class="s1"> </span><b><i>@GET</i></b></div>
<div class="p2">
<span class="s1"> </span><span class="s2"><b><i>@Path</i></b></span><span class="s3">(</span>"/json/record/{identifier}"<span class="s3">)</span><span class="s1"> </span></div>
<div class="p3">
<span class="s2"><b><i>@Produces</i></b></span><span class="s3">(</span><span class="s4"><b>MediaType</b></span><span class="s3">.</span><b><i>APPLICATION_JSON</i></b><span class="s3">)</span></div>
<div class="p3">
<span class="s5"><b>public</b></span> <span class="s4"><b>String</b></span> <span class="s3"><b>getRecordJSON</b>(</span><span class="s2"><b><i>@PathParam</i></b></span><span class="s3">(</span><span class="s6">"identifier"</span><span class="s3">)</span><span class="s4"><b>String</b></span> identifier<span class="s3">)</span> <span class="s3">{</span><span class="Apple-tab-span"> </span> </div>
<div class="p3">
<span class="s5"><b>return</b></span> <span class="s4"><b>Integer</b></span><span class="s3">.</span><i>toString</i><span class="s3">(</span><span class="s4">read</span><span class="s3">(</span>identifier<span class="s3">).</span>size<span class="s3">());</span></div>
<div class="p4">
<span class="s7">
</span></div>
<div class="p3">
<span class="s3">}</span></div>
</div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-90733305004444403602016-04-29T21:26:00.002-07:002016-12-18T16:36:19.638-08:00VM creation for Microservices HA infrastructureAmazon EC2, RDS and ElasticBeanstalk work very well but are a bit expensive when you have your own 12 core systems at home that can be used for most backend services.<br />
<br />
<br />
<h3>
VM Boostrap</h3>
<br />
Normally I would use Chef to provision the VM. This is the manual configuration list to get a repeatable VM snapshot built.<br />
<br />
We are running a RHEL 7.3 VM on top of either Windows 10 or MacOS 10<br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: 13.2px; line-height: 18.48px;"><br /></span>
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: 13.2px; line-height: 18.48px;">Register System</span><br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: 13.2px; line-height: 18.48px;">Attach Subscription</span><br />
<br />
1. Change Networks and hostname (some services need differentiation in an HA environment)<br />
[root@localhost ~]# vi /etc/hosts<br />
[rootRHEL7@localhost ~]# hostnamectl set-hostname biometric-vm01<br />
[rootRHEL6@localhost ~]# vi /etc/sysconfig/network<br />
[root@localhost ~]# systemctl restart systemd-hostnamed<br />
<div>
<br /></div>
<pre class="highlight plaintext" style="background: rgb(238, 238, 238); border-radius: 4px; border: none; box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 20px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 10px 15px; word-break: break-all; word-wrap: normal;"><span style="white-space: pre-wrap;">verify /etc/hosts has the correct ip set for the VM</span></pre>
<pre class="highlight plaintext" style="background: rgb(238, 238, 238); border-radius: 4px; border: none; box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 20px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 10px 15px; word-break: break-all; word-wrap: normal;"><span style="white-space: pre-wrap;"> If you have 2 NIC's in this case the wired nic is on a private internal subnet with no internet access and the wireless nic is our internet access - you will need to change the METRIC order. You need to set the internet live nic as the first in the routing sequence by setting the METRIC lowest on wireless and turning off DEFROUTE, IPV4_DEFROUTE, PEERDNS etc on the private wired nic.</span></pre>
<pre class="highlight plaintext" style="background: rgb(238, 238, 238); border-radius: 4px; border: none; box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 20px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 10px 15px; word-break: break-all; word-wrap: normal;"><span style="white-space: pre-wrap;">Check regenerated
[root@nuc14-i3 ~]# cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 192.168.100.1
</span></pre>
<pre class="highlight plaintext" style="background: rgb(238, 238, 238); border-radius: 4px; border: none; box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 20px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 10px 15px; word-break: break-all; word-wrap: normal;"><span style="white-space: pre-wrap;">Make the non-internet nic secondary
[root@nuc14-i3 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s25
TYPE="Ethernet"
BOOTPROTO="dhcp"
DEFROUTE=no
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="no"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="enp0s25"
UUID="938325b5-6c08-49eb-8044-8fd5dfe931fe"
DEVICE="enp0s25"
ONBOOT="yes"
PEERDNS=no
PEERROUTES=yes
IPV6_PEERDNS=no
IPV6_PEERROUTES=yes
METRIC=200</span></pre>
<br />
2. Install Java 8<br />
[root@biometric-vm01 ~]# alternatives --config java<br />
Selection Command<br />
-----------------------------------------------<br />
1 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64/jre/bin/java<br />
*+ 2 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.65-3.b17.el7.x86_64/jre/bin/java<br />
<br />
Ideally install the SUN/Oracle JDK<br />
download from <a href="http://java.sun.com/">http://java.sun.com</a><br />
<br />
<br />
3. yum update -y<br />
<div>
<br /></div>
4. Install Tomcat (In the past i would install a full EE server like WebLogic 12c but a Tomcat + Spring container is now the way forward)<br />
<br />
<div class="p1">
<span class="s1"> firewall-cmd --permanent --zone=public --add-port=8080/tcp</span></div>
<div class="p1">
<span class="s1"> sudo groupadd tomcat</span></div>
<div class="p1">
<span class="s1"> sudo useradd -M -s /bin/nologin -g tomcat -d /opt/tomcat tomcat</span></div>
<div class="p1">
<span class="s1"> cd ~</span></div>
<div class="p1">
<span class="s1"> http://apache.mirror.rafal.ca/tomcat/tomcat-8/v8.0.39/bin/apache-tomcat-8.0.39.tar.gz</span></div>
<div class="p1">
<span class="s1"> sudo mkdir /opt/tomcat</span></div>
<div class="p1">
<span class="s1"> sudo tar xvf apache-tomcat-8*tar.gz -C /opt/tomcat --strip-components=1</span></div>
<div class="p1">
<span class="s1"> cd /opt/tomcat</span></div>
<div class="p1">
<span class="s1"> sudo chgrp -R tomcat conf</span></div>
<div class="p1">
<span class="s1"> sudo chmod g+rwx conf</span></div>
<div class="p1">
<span class="s1"> sudo chmod g+r conf/*</span><br />
<span class="s1"> cd /opt</span></div>
<div class="p1">
<span class="s1"> sudo chown -R tomcat tomcat/</span></div>
<div class="p1">
<span class="s1"> sudo vi /etc/systemd/system/tomcat.service</span><br />
<span class="s1"><br /></span>
<span class="s1"></span><br />
<pre class=" language-bash" style="background: rgb(39, 40, 34); border-radius: 0.3em; box-sizing: border-box; color: #f8f8f2; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 16px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-wrap: normal;"><code class=" language-bash" style="background: none; box-sizing: border-box; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 1em; line-height: 1.5; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-spacing: normal; word-wrap: normal;"><span class="token punctuation" style="box-sizing: border-box;">[</span>Unit<span class="token punctuation" style="box-sizing: border-box;">]</span>
Description<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>Apache Tomcat8
After<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>syslog.target network.target</code></pre>
<pre class=" language-bash" style="background: rgb(39, 40, 34); border-radius: 0.3em; box-sizing: border-box; color: #f8f8f2; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 16px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-wrap: normal;"><code class=" language-bash" style="background: none; box-sizing: border-box; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 1em; line-height: 1.5; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-spacing: normal; word-wrap: normal;"><span class="token punctuation" style="box-sizing: border-box;">[</span>Service<span class="token punctuation" style="box-sizing: border-box;">]
</span> Type<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>forking
User<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>tomcat
Group<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>tomcat
</code> Environment<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box; font-size: 1em; word-spacing: normal;">=</span><span style="background-color: initial; font-size: 1em; word-spacing: normal;">JAVA_HOME</span><span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box; font-size: 1em; word-spacing: normal;">=</span><span style="background-color: initial; font-size: 1em; word-spacing: normal;">/usr/lib/jvm/jre</span></pre>
<pre class=" language-bash" style="background: rgb(39, 40, 34); border-radius: 0.3em; box-sizing: border-box; color: #f8f8f2; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 16px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-wrap: normal;"><code class=" language-bash" style="background: none; box-sizing: border-box; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 1em; line-height: 1.5; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-spacing: normal; word-wrap: normal;"> Environment<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>CATALINA_PID<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>/opt/tomcat/temp/tomcat.pid
Environment<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>CATALINA_HOME<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>/opt/tomcat
Environment<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>CATALINA_BASE<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>/opt/tomcat
Environment<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span><span class="token string" style="box-sizing: border-box; color: #a6e22e;">'CATALINA_OPTS=-Xms512M -Xmx4096M -server -XX:+UseParallelGC'</span>
Environment<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span><span class="token string" style="box-sizing: border-box; color: #a6e22e;">'JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom'
</span> ExecStart<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>/opt/tomcat/bin/startup.sh
ExecStop<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>/bin/kill -15 <span class="token variable" style="box-sizing: border-box;">$MAINPID</span></code></pre>
<pre class=" language-bash" style="background: rgb(39, 40, 34); border-radius: 0.3em; box-sizing: border-box; color: #f8f8f2; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 16px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-wrap: normal;"><code class=" language-bash" style="background: none; box-sizing: border-box; font-family: Consolas, Monaco, "Andale Mono", "Ubuntu Mono", monospace; font-size: 1em; line-height: 1.5; tab-size: 4; text-shadow: rgba(0, 0, 0, 0.298039) 0px 1px; word-break: normal; word-spacing: normal; word-wrap: normal;"><span class="token punctuation" style="box-sizing: border-box;">[</span>Install<span class="token punctuation" style="box-sizing: border-box;">]</span>
WantedBy<span class="token operator" style="background: rgba(255, 255, 255, 0.498039); box-sizing: border-box;">=</span>multi-user.target</code></pre>
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1"> 72 sudo systemctl daemon-reload</span></div>
<div class="p1">
<span class="s1"> 73 sudo systemctl start tomcat</span></div>
<div class="p1">
<span class="s1"> 74 sudo systemctl enable tomcat</span></div>
<br />
<div class="p1">
<span class="s1"> 81 systemctl restart firewalld.service</span><br />
<span class="s1"><br /></span>
[root@rhel73os0prod opt]# systemctl status tomcat.service<br />
● tomcat.service - Apache Tomcat8<br />
Loaded: loaded (/etc/systemd/system/tomcat.service; disabled; vendor preset: disabled)<br />
Active: active (running) since Sat 2016-12-10 22:07:05 EST; 12s ago<br />
Process: 4731 ExecStart=/opt/tomcat/bin/startup.sh (code=exited, status=0/SUCCESS)<br />
Main PID: 4738 (java)<br />
CGroup: /system.slice/tomcat.service<br />
└─4738 /usr/lib/jvm/jre/bin/java -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties -Djava...<br />
<br />
Dec 10 22:07:05 rhel73os0prod systemd[1]: Starting Apache Tomcat8...<br />
<span class="s1"></span><br />
Dec 10 22:07:05 rhel73os0prod systemd[1]: Started Apache Tomcat8.<br />
<div>
<br /></div>
</div>
<br />
5. Setup developer services<br />
<br />
6. PostgreSQL install - for ORM services via Hibernate or better EclipseLink<br />
<br />
7. MySQL install<br />
<br />
<div class="p1" style="background-color: white; color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13.2px; line-height: 18.48px;">
<a href="http://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html" style="color: #888888; text-decoration: none;">http://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html </a><br />
cd Downloads<br />
yum repolist enabled | grep "mysql.*-community.*"<br />
yum localinstall mysql57-community-release-el7-8.noarch.rpm<br />
yum install mysql-community-server</div>
Installing : mysql-community-common-5.7.12-1.el7.x86_64 1/6<br />
Installing : mysql-community-libs-5.7.12-1.el7.x86_64 2/6<br />
Installing : mysql-community-client-5.7.12-1.el7.x86_64 3/6<br />
Installing : mysql-community-server-5.7.12-1.el7.x86_64 4/6<br />
Installing : mysql-community-libs-compat-5.7.12-1.el7.x86_64 5/6<br />
Erasing : 1:mariadb-libs-5.5.47-1.el7_2.x86_64 6/6<br />
Verifying : mysql-community-client-5.7.12-1.el7.x86_64 1/6<br />
Verifying : mysql-community-server-5.7.12-1.el7.x86_64 2/6<br />
Verifying : mysql-community-common-5.7.12-1.el7.x86_64 3/6<br />
Verifying : mysql-community-libs-5.7.12-1.el7.x86_64 4/6<br />
Verifying : mysql-community-libs-compat-5.7.12-1.el7.x86_64 5/6<br />
Verifying : 1:mariadb-libs-5.5.47-1.el7_2.x86_64 6/6<br />
<br />
<div class="p1" style="background-color: white; color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13.2px; line-height: 18.48px;">
service mysqld start<br />
grep 'temporary password' /var/log/mysqld.log<br />
mysql -uroot -p<br />
mysql> ALTER User 'root'@'localhost' IDENTIFIED BY 'S';<br />
/usr/bin/mysql_secure_installation<br />
mysql -u root -p<br />
<br />
<div class="crayon-line" id="crayon-571b8c147a837781963048-2">
mysql<span class="crayon-h">></span><span class="crayon-h">create database biometric;</span><span class="crayon-st"></span><span class="crayon-h"></span><span class="crayon-r"></span><span class="crayon-h"></span><br />
<span class="crayon-h"><br /></span>
<span class="crayon-h">here</span><br />
<br /></div>
mysql>create user<span class="crayon-r"></span><span class="crayon-h"> </span><span class="crayon-s">'obrienlabs'</span>@<span class="crayon-s">'192.168.0.21'</span><span class="crayon-h"> </span>IDENTIFIED<span class="crayon-h"> </span><span class="crayon-r">BY</span><span class="crayon-h"> </span><span class="crayon-s">'OC'</span>;</div>
<div class="p1" style="background-color: white; color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13.2px; line-height: 18.48px;">
mysql<span class="crayon-h">></span>grant all on<span class="crayon-r"></span><span class="crayon-h"> biometric</span>.*<span class="crayon-h"> to</span><span class="crayon-r"></span><span class="crayon-h"> </span><span class="crayon-s">'obrienlabs'</span>@<span class="crayon-s">'192.168.0.21'</span>;<br />
mysql> create user 'obriensystems'@'%' IDENTIFIED BY 'S';<br />
mysql> grant all on biometric.* to 'obriensystems'@'%';<br />
mysql<span class="crayon-h">></span><span class="crayon-h">flush privileges</span>;</div>
<div class="p1" style="background-color: white; color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13.2px; line-height: 18.48px;">
firewall-cmd --permanent --zone=public --add-service=mysql<br />
firewall-cmd --permanent --zone=public --add-port=3306/tcp<br />
systemctl restart firewalld.service<br />
<br />
help hibernate run the following<br />
<div class="p1">
<span class="s1"><b>private</b></span> <span class="s2">TypedQuery</span><span class="s3"><</span>Record<span class="s3">></span> <span class="s3"><b>getTypedCriteriaQuery</b>(</span><span class="s2">EntityManager</span> entityManager<span class="s3">,</span> <span class="s2"><b>String</b></span> user<span class="s3">)</span> <span class="s3">{</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">CriteriaBuilder</span> <b>cb</b> <span class="s3">=</span> entityManager<span class="s3">.</span>getCriteriaBuilder<span class="s3">();</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">CriteriaQuery</span><span class="s3"><</span>Record<span class="s3">></span> <b>query</b> <span class="s3">=</span> cb<span class="s3">.</span>createQuery<span class="s3">(</span><span class="s2"><b>Record</b></span><span class="s3">.</span><span class="s1"><b>class</b></span><span class="s3">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">Root</span><span class="s3"><</span>Record<span class="s3">></span> <b>target</b> <span class="s3">=</span> query<span class="s3">.</span>from<span class="s3">(</span><span class="s2"><b>Record</b></span><span class="s3">.</span><span class="s1"><b>class</b></span><span class="s3">);</span></div>
<div class="p2">
<span class="s4"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>// workaround for http://stackoverflow.com/questions/16348354/how-do-i-write-a-max-query-with-a-where-clause-in-jpa-2-0</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">SingularAttribute</span><span class="s3"><?</span> <span class="s1"><b>super</b></span> <span class="s2"><b>Record</b></span><span class="s3">,</span> Long<span class="s3">></span> <b>anAttribute</b> <span class="s3">=</span> entityManager<span class="s3">.</span>getMetamodel<span class="s3">()</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s3">.</span>entity<span class="s3">(</span><span class="s2"><b>Record</b></span><span class="s3">.</span><span class="s1"><b>class</b></span><span class="s3">).</span>getSingularAttribute<span class="s3">(</span><span class="s5">"tsStop"</span><span class="s3">,</span> <span class="s2"><b>Long</b></span><span class="s3">.</span><span class="s1"><b>class</b></span><span class="s3">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>query<span class="s3">.</span>orderBy<span class="s3">(</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>cb<span class="s3">.</span>desc<span class="s3">(</span>target<span class="s3">.</span>get<span class="s3">(</span>anAttribute<span class="s3">)));</span><br />
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>query<span class="s1">.</span>where<span class="s1">(</span></div>
<span class="s3">
</span><br />
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>cb<span class="s1">.</span>equal<span class="s1">(</span>target<span class="s1">.</span>get<span class="s1">(</span><span class="s2">"userId"</span><span class="s1">),</span> user<span class="s1">));</span></div>
</div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s2">TypedQuery</span><span class="s3"><</span>Record<span class="s3">></span> <b>typedQuery</b> <span class="s3">=</span> entityManager<span class="s3">.</span>createQuery<span class="s3">(</span>query<span class="s3">);</span></div>
<div class="p1">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1"><b>return</b></span> typedQuery<span class="s3">;</span><span class="Apple-tab-span"> </span></div>
<br />
<div class="p1">
<span class="s3">}</span></div>
<br />
#https://github.com/Elgg/Elgg/issues/8121<br />
# remove ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES<br />
SET GLOBAL sql_mode = 'NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';<br />
SET SESSION sql_mode = 'NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';<br />
<br />
SELECT @@GLOBAL.sql_mode;<br />
SELECT @@SESSION.sql_mode;</div>
8. Neo4J install - for graph based nosql (a geohash tree for example)<br />
<br />
<br />
<h3>
9. JPA</h3>
<h4>
Hibernate on Spring Repository</h4>
<div>
<pre><div>
<?xml version="1.0"?></div>
<div>
<beans xmlns="http://www.springframework.org/schema/beans"</div>
<div>
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"</div>
<div>
xmlns:task="http://www.springframework.org/schema/task"</div>
<div>
xmlns:context="http://www.springframework.org/schema/context"</div>
<div>
xmlns:tx="http://www.springframework.org/schema/tx"</div>
<div>
xmlns:p="http://www.springframework.org/schema/p"</div>
<div>
xmlns:util="http://www.springframework.org/schema/util"</div>
<div>
xmlns:mvc="http://www.springframework.org/schema/mvc"</div>
<div>
xsi:schemaLocation="http://www.springframework.org/schema/beans</div>
<div>
http://www.springframework.org/schema/beans/spring-beans.xsd</div>
<div>
http://www.springframework.org/schema/context</div>
<div>
http://www.springframework.org/schema/context/spring-context.xsd</div>
<div>
http://www.springframework.org/schema/task</div>
<div>
http://www.springframework.org/schema/task/spring-task.xsd</div>
<div>
http://www.springframework.org/schema/mvc</div>
<div>
http://www.springframework.org/schema/mvc/spring-mvc.xsd</div>
<div>
http://www.springframework.org/schema/tx</div>
<div>
http://www.springframework.org/schema/tx/spring-tx.xsd</div>
<div>
http://www.springframework.org/schema/util</div>
<div>
http://www.springframework.org/schema/util/spring-util.xsd"></div>
<div>
</div>
<div>
<context:component-scan base-package="com....orm"/></div>
<div>
<context:annotation-config/></div>
<div>
</div>
<div>
<!-- mvc:annotation-driven/--></div>
<div>
<!-- tx:annotation-driven/--></div>
<div>
</div>
<div>
<!-- bean id="contextAwareMBeanServer" class="com.common.DefaultServerMBean" /></div>
<div>
</div>
<div>
<bean id="dbConfig" class="com.common.util.NspConfig" depends-on="contextAwareMBeanServer"></div>
<div>
<constructor-arg type="java.lang.String" index="0" name="aInFileName" value="pgsql.conf" /></div>
<div>
</bean--></div>
<div>
</div>
<div>
<!-- bean class="com..app.ts.bwcal.config.SwaggerConfig"/--></div>
<div>
</div>
<div>
<bean class="org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor"/></div>
<div>
<bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor"/></div>
<div>
</div>
<div>
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close"></div>
<div>
<!-- bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource" destroy-method="close"--> </div>
<div>
<property name="driverClass" value="org.postgresql.Driver" /></div>
<div>
<property name="jdbcUrl" value="jdbc:postgresql://127.0.0.1:6432/orbwdb?ssl=true&amp;sslfactory=org.postgresql.ssl.NonValidatingFactory" /></div>
<div>
<property name="user" value="taskscheduler" /></div>
<div>
<property name="password" value="taskscheduler" /></div>
<div>
<property name="minPoolSize" value="10"/></div>
<div>
<property name="maxPoolSize" value="20"/></div>
<div>
<property name="acquireIncrement" value="2"/></div>
<div>
<property name="maxStatementsPerConnection" value="100"/></div>
<div>
<property name="testConnectionOnCheckin" value="true" /></div>
<div>
<property name="testConnectionOnCheckout" value="true" /></div>
<div>
<property name="idleConnectionTestPeriod" value="90" /></div>
<div>
<property name="preferredTestQuery" value="SELECT 1" /> </div>
<div>
</bean></div>
<div>
</div>
<div>
<!-- bean id="liquibase2" class="liquibase.integration.spring.SpringLiquibase"></div>
<div>
<property name="dataSource" ref="dataSource" /></div>
<div>
<property name="changeLog" value="classpath:db/changelog/db-changelog-master.xml" /></div>
<div>
<property name="defaultSchema" value="taskscheduler" /></div>
<div>
</bean--> </div>
<div>
</div>
<div>
<bean id="jpaAdapter" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"></div>
<div>
<property name="databasePlatform" value="org.hibernate.dialect.PostgreSQLDialect"/></div>
<div>
<property name="generateDdl" value="true"/></div>
<div>
<property name="showSql" value="true"/></div>
<div>
</bean></div>
<div>
</div>
<div>
<bean id="emf" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"</div>
<div>
p:jpaVendorAdapter-ref="jpaAdapter"></div>
<div>
<property name="persistenceUnitName" value="openflow"></property></div>
<div>
<property name="dataSource" ref="dataSource"></property></div>
<div>
<property name="jpaProperties"></div>
<div>
<props></div>
<div>
<prop key="hibernate.hbm2ddl.auto">create-drop</prop></div>
<div>
<prop key="hibernate.flushMode">FLUSH_AUTO</prop></div>
<div>
<prop key="hibernate.temp.use_jdbc_metadata_defaults">false</prop></div>
<div>
</props></div>
<div>
</property></div>
<div>
</bean></div>
<div>
</div>
<div>
<bean id="txManager" class="org.springframework.orm.jpa.JpaTransactionManager"</div>
<div>
p:entityManagerFactory-ref="emf"/></div>
<div>
</div>
<div>
<tx:annotation-driven transaction-manager="txManager"/></div>
<div>
</div>
<div>
</beans></div>
</pre>
</div>
<h4>
EclipseLink on Spring Repository</h4>
<h4>
Hibernate on Spring-data-jpa</h4>
<h4>
EclipseLink on Spring-data-jpa</h4>
<br />Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-81571350815693620352015-12-15T11:26:00.001-08:002016-01-30T07:08:05.014-08:00Historical Links - for Research<a href="http://eclipse.1072660.n5.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=357">http://eclipse.1072660.n5.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=357</a><br />
<a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=266912">https://bugs.eclipse.org/bugs/show_bug.cgi?id=266912</a><br />
<a href="http://wiki.eclipse.org/User:Michael.obrien.oracle.com">http://wiki.eclipse.org/User:Michael.obrien.oracle.com</a><br />
<a href="http://wiki.eclipse.org/User:Michael.f.obrien.eclipselink.org">http://wiki.eclipse.org/User:Michael.f.obrien.eclipselink.org</a><br />
<a href="https://www.linkedin.com/in/michael-o-brien-95b65b32">https://www.linkedin.com/in/michael-o-brien-95b65b32</a><br />
<a href="https://www.quora.com/profile/Michael-F-OBrien">https://www.quora.com/profile/Michael-F-OBrien</a><br />
<a href="https://community.oracle.com/thread/2249518?tstart=0">https://community.oracle.com/thread/2249518?tstart=0</a><br />
<a href="https://community.oracle.com/people/872988?customTheme=otn">https://community.oracle.com/people/872988?customTheme=otn</a><br />
<a href="https://netbeans.org/projects/platform/lists/dev/archive/2011-01/message/399">https://netbeans.org/projects/platform/lists/dev/archive/2011-01/message/399</a><br />
<a href="http://forums.terracotta.org/forums/posts/list/2227.page">http://forums.terracotta.org/forums/posts/list/2227.page</a><br />
<a href="http://stackoverflow.com/users/417402/michael-obrien">http://stackoverflow.com/users/417402/michael-obrien</a><br />
<a href="https://jira.spring.io/si/jira.issueviews:issue-html/SPR-6826/SPR-6826.html">https://jira.spring.io/si/jira.issueviews:issue-html/SPR-6826/SPR-6826.html</a><br />
<a href="http://www.theserverside.com/news/thread.tss?thread_id=50305#271037">http://www.theserverside.com/news/thread.tss?thread_id=50305#271037</a><br />
<a href="http://wiki.eclipse.org/EclipseLink/Examples/JPA/WebLogic_Web_Tutorial">http://wiki.eclipse.org/EclipseLink/Examples/JPA/WebLogic_Web_Tutorial</a><br />
<a href="https://docs.oracle.com/cd/E28280_01/apirefs.1111/b32476/oracle/toplink/platform/server/oc4j/Oc4j_11_1_1_Platform.htm">https://docs.oracle.com/cd/E28280_01/apirefs.1111/b32476/oracle/toplink/platform/server/oc4j/Oc4j_11_1_1_Platform.htm</a>l<br />
<a href="https://community.oracle.com/thread/3544823?start=0&tstart=0">https://community.oracle.com/thread/3544823?start=0&tstart=0</a><br />
<a href="http://www.objectivej.com/">http://www.objectivej.com/</a><br />
<div>
<br />
My Research<br />
Weather radar capture and processing for thunderstorm tracking prediction<br />
<a href="http://obrienscience.blogspot.ca/2011/08/radar.html">http://obrienscience.blogspot.ca/2011/08/radar.html</a><br />
<br />
Parallel processing throughput optimization<br />
<a href="http://obrienscience.blogspot.ca/2011/03/distributed-and-multithreaded.html">http://obrienscience.blogspot.ca/2011/03/distributed-and-multithreaded.html</a><br />
<a href="http://obrienscience.blogspot.ca/2011/01/sequences-and-patterns.html">http://obrienscience.blogspot.ca/2011/01/sequences-and-patterns.html</a><br />
<br />
<br />
realtime biometric (heartrate) tracking for detection of exercise induced HR abnormalities<br />
<a href="http://eclipsejpa.blogspot.ca/2013/09/restjax-rs-ios7-weblogic-12c-bluetooth.html">http://eclipsejpa.blogspot.ca/2013/09/restjax-rs-ios7-weblogic-12c-bluetooth.html</a></div>
<div>
<br /></div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-36193138414372977162015-11-05T16:51:00.002-08:002017-12-16T19:28:59.908-08:00DevOps<h3>
</h3>
<h3>
Docker</h3>
<div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 18.0px Menlo; color: #ffffff; background-color: #173b79; min-height: 21.0px}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 18.0px Menlo; color: #ffffff; background-color: #173b79}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<div class="p1">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1"></span><br /></div>
<div class="p2">
<span class="s1"><span class="Apple-converted-space"> </span>docker login</span></div>
<div class="p2">
<span class="s1"><span class="Apple-converted-space"> </span>vi Dockerfile<span class="Apple-converted-space"> </span></span></div>
<div class="p2">
<span class="s1"><span class="Apple-converted-space"> </span>docker build -t radarweather .</span></div>
<div class="p2">
<span class="s1"><span class="Apple-converted-space"> </span>docker run radarweather:latest</span></div>
<div class="p2">
<span class="s1"><span class="Apple-converted-space"> </span>docker tag d7fd00adce4e michaelobrien/radarweather</span></div>
<div class="p2">
<span class="s1"><span class="Apple-converted-space"> </span>docker push michaelobrien/radarweather</span></div>
</div>
<h3>
<br /></h3>
<h3>
Switch JDK on MacOS</h3>
<div class="p1">
<span class="s1">obrienlabs-mbp15:bin michaelobrien$ /usr/libexec/java_home -V</span></div>
<div class="p1">
<span class="s1">Matching Java Virtual Machines (6):</span></div>
<div class="p1">
<span class="s1"> 1.8.0_65, x86_64:<span class="Apple-tab-span"> </span>"Java SE 8"<span class="Apple-tab-span"> </span>/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home</span></div>
<div class="p1">
<span class="s1"> 1.8.0_20, x86_64:<span class="Apple-tab-span"> </span>"Java SE 8"<span class="Apple-tab-span"> </span>/Library/Java/JavaVirtualMachines/jdk1.8.0_20.jdk/Contents/Home</span></div>
<div class="p1">
<span class="s1"> 1.8.0_11, x86_64:<span class="Apple-tab-span"> </span>"Java SE 8"<span class="Apple-tab-span"> </span>/Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home</span></div>
<div class="p1">
<span class="s1"> 1.7.0_51, x86_64:<span class="Apple-tab-span"> </span>"Java SE 7"<span class="Apple-tab-span"> </span>/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home</span></div>
<div class="p1">
<span class="s1"> 1.7.0_45, x86_64:<span class="Apple-tab-span"> </span>"Java SE 7"<span class="Apple-tab-span"> </span>/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home</span></div>
<div class="p1">
<span class="s1"> 1.7.0_15, x86_64:<span class="Apple-tab-span"> </span>"Java SE 7"<span class="Apple-tab-span"> </span>/Library/Java/JavaVirtualMachines/jdk1.7.0_15.jdk/Contents/Home</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">obrienlabs-mbp15:bin michaelobrien$ sudo ln -nsf /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents \</span></div>
<div class="p1">
<span class="s1">> /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK</span></div>
<div class="p1">
<span class="s1">ln: /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK: Operation not permitted</span></div>
<div class="p1">
<span class="s1">obrienlabs-mbp15:bin michaelobrien$ export JAVA_HOME=`/usr/libexec/java_home -v 1.8.0_65`</span></div>
<div class="p1">
<span class="s1">obrienlabs-mbp15:bin michaelobrien$ echo $JAVA_HOME</span></div>
<div class="p1">
<span class="s1">/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home</span></div>
<div class="p1">
<span class="s1">obrienlabs-mbp15:bin michaelobrien$ vi ~/.bashrc</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">obrienlabs-mbp15:bin michaelobrien$ java -version</span></div>
<div class="p1">
<span class="s1">java version "1.8.0_65"</span></div>
<div class="p1">
<span class="s1">Java(TM) SE Runtime Environment (build 1.8.0_65-b17)</span></div>
<div class="p1">
<span class="s1">Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)</span></div>
<div class="p1">
<span class="s1">obrienlabs-mbp15:bin michaelobrien$ javac -version</span></div>
<br />
<div class="p1">
<span class="s1">javac 1.8.0_65</span><br />
<span class="s1"><br /></span><span class="s1">Ubuntu</span><br />
<span class="s1">Add proxy for APT</span><br />
<span class="s1"><span style="background-color: white; color: #111111; font-family: "ubuntu" , "arial" , "libra sans" , sans-serif; font-size: 15px; line-height: 19.5px;"> Add the following line to </span><code style="background-color: #eeeeee; border: 0px; color: #111111; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, sans-serif; font-size: 13px; margin: 0px; padding: 1px 5px; white-space: pre-wrap;">/etc/apt/apt.conf</code><span style="background-color: white; color: #111111; font-family: "ubuntu" , "arial" , "libra sans" , sans-serif; font-size: 15px; line-height: 19.5px;"> : </span><code style="background-color: #eeeeee; border: 0px; color: #111111; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, sans-serif; font-size: 13px; margin: 0px; padding: 1px 5px; white-space: pre-wrap;">Acquire::http::proxy "http://global.proxy.*-*.com:8000/";</code></span><br />
<br />
Static Network Config<br />
Second Network Config survives reboot<br />
RHEL<br />
>vi /etc/sysconfig/network-scripts/ifcfg-*<br />
ONBOOT=yes<br />
<br />
BOOTPROTO=static<br />
NETWORK=192.168.2.0<br />
NETMASK=255.255.255.0<br />
IPADDR=192.168.2.101<br />
USERCTL=no<br />
DNS1=192.168.2.1<br />
GATEWAY=192.168.2.1<br />
<br />
for dns check /etc/yp.conf<br />
<br />
Configure VNC<br />
<a href="https://access.redhat.com/solutions/966063">https://access.redhat.com/solutions/966063</a> <br />
<br />
<h3>
Configuring AWS CodeCommit on OSX</h3>
<div class="p1">
<a href="https://console.aws.amazon.com/codecommit/home?region=us-east-1#/repository/neo4j/browse/HEAD/--/">https://console.aws.amazon.com/codecommit/home?region=us-east-1#/repository/neo4j/browse/HEAD/--/</a></div>
<div class="p2">
<br /></div>
<ul class="ul1">
<li class="li3"><span class="s1">git config --global credential.helper '!aws codecommit credential-helper $@'</span></li>
<li class="li3"><span class="s1">git config --global credential.UseHttpPath true</span></li>
<li class="li4"><span class="s2">534 curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"</span></li>
<li class="li5"><span class="s2">535 unzip awscli-bundle.zip</span></li>
<li class="li5"><span class="s2">536 sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws</span>537<span style="font-variant-ligatures: no-common-ligatures;"> aws --version</span>539<span style="font-variant-ligatures: no-common-ligatures;"> aws configure</span>541<span style="font-variant-ligatures: no-common-ligatures;"> </span><span style="font-variant-ligatures: no-common-ligatures;">aws ec2 describe-instances --output table --region us-east-1</span></li>
</ul>
<div class="p2">
<br /></div>
<div class="p1">
verify</div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; background-color: #ffffff}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; background-color: #ffffff; min-height: 14.0px}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
li.li3 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 21.0px; font: 14.0px Courier; color: #444444; background-color: #f9f9f9}
li.li4 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 16.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
li.li5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-kerning: none}
span.s2 {font-variant-ligatures: no-common-ligatures}
ul.ul1 {list-style-type: none}
</style>
<br />
<div class="p1">
<a href="http://docs.aws.amazon.com/codecommit/latest/userguide/troubleshooting.html?icmpid=docs_acc_console_connect#troubleshooting-macoshttps">http://docs.aws.amazon.com/codecommit/latest/userguide/troubleshooting.html?icmpid=docs_acc_console_connect#troubleshooting-macoshttps</a></div>
<br />
<h3>
Eclipse IDE Workarounds</h3>
<h4>
Maven Project changes remove spring lib folder in STS</h4>
This error occurs a couple times a month when I asjust a war project - eclipse causes spring exceptions on tomcat startup. The issue is the spring jars local to the war classloader are not loaded<br />
<span style="color: #274e13; font-family: "courier new" , "courier" , monospace;"><br /></span>
<br />
<div class="p1">
<span style="color: black; font-family: "courier new" , "courier" , monospace;">Dec 24, 2016 10:21:03 AM org.apache.catalina.core.StandardContext listenerStart</span></div>
<div class="p1">
<span style="color: black; font-family: "courier new" , "courier" , monospace;">SEVERE: Error configuring application listener of class org.springframework.web.context.request.RequestContextListener</span></div>
<div class="p1">
<span style="color: black; font-family: "courier new" , "courier" , monospace;"><span class="s1">java.lang.ClassNotFoundException</span>: org.springframework.web.context.request.RequestContextListener</span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #ffa4ff}
span.s1 {text-decoration: underline ; color: #c9fa0e}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p1">
<span style="color: black; font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span"> </span>at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(<span class="s1">WebappClassLoaderBase.java:1333</span>)</span></div>
<br />
fix:<br />
Add back<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span><attribute name="org.eclipse.jst.component.dependency" value="/WEB-INF/lib"/><br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAdwiO30o-655tUOGvmtQrJdVAwrNx3R_NF7PiKTjkDUkN_ciuljrp5WDXKhoTw8dnEmmaIsqSteDzv1Z6d_-XeB0hj9c5FWRVL7VJ47-uEPjjvmAdWx4k0xup0rLvhZGmUh01klRTkD0v/s1600/Screenshot+2016-12-24+10.22.47.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAdwiO30o-655tUOGvmtQrJdVAwrNx3R_NF7PiKTjkDUkN_ciuljrp5WDXKhoTw8dnEmmaIsqSteDzv1Z6d_-XeB0hj9c5FWRVL7VJ47-uEPjjvmAdWx4k0xup0rLvhZGmUh01klRTkD0v/s1600/Screenshot+2016-12-24+10.22.47.png" /></a></div>
<div>
<br /></div>
<br />
<h3>
<span class="s1">IOS 9.0/Xcode 7 Security changes force HTTPS</span></h3>
<div class="p1">
<span class="s1">state</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Xcode 7 changes block http access</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">2015-10-17 19:40:27.422 Biometric[72723:1193943] App Transport Security has blocked a cleartext HTTP (http://) resource load since it is insecure. Temporary exceptions can be configured via your app's Info.plist file.</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<span class="s1">
</span><br />
<div class="p1">
<span class="s1"><a href="https://developer.apple.com/library/prerelease/ios/technotes/App-Transport-Security-Technote/index.htm">https://developer.apple.com/library/prerelease/ios/technotes/App-Transport-Security-Technote/index.htm</a>l</span><br />
<span class="s1"><br /></span>
<br />
<h3>
<span class="s1">PostgreSQL 9.4 Installation - RHEL 7.2</span></h3>
<div class="p1">
<span class="s1"> yum install http://yum.postgresql.org/9.4/redhat/rhel-7-x86_64/pgdg-redhat94-9.4-1.noarch.rpm</span></div>
<div class="p1">
<span class="s1"> yum install postgresql94-server postgresql94-contrib</span></div>
<div class="p1">
<span class="s1"> /usr/pgsql-9.4/bin/postgresql94-setup initdb</span></div>
<div class="p1">
<span class="s1"> chkconfig postgresql-9.4 on</span></div>
<div class="p1">
<span class="s1"> systemctl enable postgresql-9.4.service</span></div>
<span class="s1">
</span><br />
<div class="p1">
<span class="s1"> systemctl start postgresql-9.4.service</span><br />
<span class="s1"><br /></span>
<span class="s1">
</span><br />
<div class="p1">
<span class="s1">vi /var/lib/pgsql/9.4/data/pg_hba.conf </span></div>
<div class="p1">
<span class="s1">host all all 192.168.0.0/24 trust</span></div>
<span class="s1">
</span><br />
<div class="p1">
<br />
<h3>
MySQL 5.7 install - RHEL 7.2</h3>
<a href="http://dev.mysql.com/downloads/mysql/">http://dev.mysql.com/downloads/mysql/ </a><br />
<a href="http://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html">http://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html </a><br />
cd Downloads<br />
yum repolist enabled | grep "mysql.*-community.*"<br />
yum localinstall mysql57-community-release-el7-8.noarch.rpm <br />
yum install mysql-community-server<br />
service mysqld start<br />
grep 'temporary password' /var/log/mysqld.log<br />
mysql -uroot -p <br />
mysql> ALTER User 'root'@'localhost' IDENTIFIED BY 'S';<br />
/usr/bin/mysql_secure_installation <br />
mysql -u root -p<br />
<br />
<div class="crayon-line" id="crayon-571b8c147a837781963048-2">
mysql<span class="crayon-h">></span><span class="crayon-h">create database biometric;</span><span class="crayon-st"></span><span class="crayon-h"></span><span class="crayon-r"></span><span class="crayon-h"></span></div>
mysql>create user<span class="crayon-r"></span><span class="crayon-h"> </span><span class="crayon-s">'obrienlabs'</span>@<span class="crayon-s">'192.168.0.21'</span><span class="crayon-h"> </span>IDENTIFIED<span class="crayon-h"> </span><span class="crayon-r">BY</span><span class="crayon-h"> </span><span class="crayon-s">'OC'</span>;</div>
<div class="p1">
mysql<span class="crayon-h">></span>grant all on<span class="crayon-r"></span><span class="crayon-h"> biometric</span>.*<span class="crayon-h"> to</span><span class="crayon-r"></span><span class="crayon-h"> </span><span class="crayon-s">'obrienlabs'</span>@<span class="crayon-s">'192.168.0.21'</span>;<br />
mysql> create user 'obriensystems'@'%' IDENTIFIED BY 'S';<br />
mysql> grant all on biometric.* to 'obriensystems'@'%';<br />
mysql<span class="crayon-h">></span><span class="crayon-h">flush privileges</span>;</div>
<div class="p1">
firewall-cmd --permanent --zone=public --add-service=mysql<br />
firewall-cmd --permanent --zone=public --add-port=3306/tcp<br />
systemctl restart firewalld.service<br />
<br />
help hibernate<br />
#https://github.com/Elgg/Elgg/issues/8121<br />
# remove ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES<br />
SET GLOBAL sql_mode = 'NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';<br />
SET SESSION sql_mode = 'NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';<br />
<br />
SELECT @@GLOBAL.sql_mode;<br />
SELECT @@SESSION.sql_mode;<br />
<br />
<br />
block systems doing 330ms scans on /manager<br />
24-Apr-2016 20:56:23.562 WARNING [http-nio-8080-exec-1] org.apache.catalina.realm.LockOutRealm.authenticate An attempt was made to authenticate the locked user "manager"<br />
191.100.28.217 - - [24/Apr/2016:20:56:23 -0400] "GET /manager/html HTTP/1.1" 401 2473 <br />
<div class="p1">
<span class="s1">traceroute to 191.100.28.217 (191.100.28.217), 30 hops max, 60 byte packets</span></div>
<div class="p1">
<span class="s1"> 1 gateway (192.168.0.1) 1.751 ms 2.171 ms 2.460 ms</span></div>
<div class="p1">
<span class="s1"> 2 7.11.161.65 (7.11.161.65) 15.363 ms 15.520 ms 18.285 ms</span></div>
<div class="p1">
<span class="s1"> 3 67.231.220.65 (67.231.220.65) 19.414 ms 19.632 ms 19.744 ms</span></div>
<div class="p1">
<span class="s1"> 4 van58-9-231-73.dynamic.rogerstelecom.net (209.148.231.73) 27.127 ms 27.336 ms 35.187 ms</span></div>
<div class="p1">
<span class="s1"> 5 van58-9-229-225.dynamic.rogerstelecom.net (209.148.229.225) 38.660 ms 39.272 ms 39.359 ms</span></div>
<div class="p1">
<span class="s1"> 6 24.156.144.178 (24.156.144.178) 50.564 ms 33.076 ms 38.590 ms</span></div>
<div class="p1">
<span class="s1"> 7 if-ae-12-0.tcore2.CT8-Chicago.as6453.net (64.86.79.89) 50.078 ms 51.630 ms 51.746 ms</span></div>
<div class="p1">
<span class="s1"> 8 if-ae-26-2.tcore2.NTO-New-York.as6453.net (216.6.81.28) 58.778 ms 58.646 ms 58.857 ms</span></div>
<div class="p1">
<span class="s1"> 9 if-ae-14-15.thar2.NJY-Newark.as6453.net (66.198.111.8) 58.387 ms if-ae-26-2.tcore2.NTO-New-York.as6453.net (216.6.81.28) 66.430 ms if-ae-18-2.thar2.NJY-Newark.as6453.net (66.198.111.6) 66.812 ms</span></div>
<div class="p1">
<span class="s1">10 if-ae-11-2.tcore2.AEQ-Ashburn.as6453.net (216.6.87.137) 68.125 ms 69.242 ms if-ae-14-14.thar2.NJY-Newark.as6453.net (66.198.111.125) 67.023 ms</span></div>
<div class="p1">
<span class="s1">11 if-ae-2-2.tcore1.AEQ-Ashburn.as6453.net (216.6.87.2) 67.213 ms 67.326 ms 69.655 ms</span></div>
<div class="p1">
<span class="s1">12 if-ae-2-2.tcore1.AEQ-Ashburn.as6453.net (216.6.87.2) 56.081 ms 66.198.155.2 (66.198.155.2) 64.697 ms if-ae-2-2.tcore1.AEQ-Ashburn.as6453.net (216.6.87.2) 47.116 ms</span></div>
<div class="p1">
<span class="s1">13 66.198.155.2 (66.198.155.2) 46.881 ms te0-6-0-1-grtjaxtw1.net.telefonicaglobalsolutions.com (176.52.249.141) 71.706 ms te0-7-0-2-grtjaxtw1.net.telefonicaglobalsolutions.com (176.52.251.203) 80.713 ms</span></div>
<div class="p1">
<span class="s1">14 te0-6-0-5-grtjaxtw1.net.telefonicaglobalsolutions.com (84.16.15.199) 75.481 ms telconet_te-0-0-0-36-grtjaxtw1.net.telefonicaglobalsolutions.com (213.140.39.109) 139.924 ms 150.283 ms</span></div>
<div class="p1">
<span class="s1">15 telconet_te-0-0-0-36-grtjaxtw1.net.telefonicaglobalsolutions.com (213.140.39.109) 147.605 ms * *</span></div>
<div class="p1">
<span class="s1">16 * 186.3.125.54 (186.3.125.54) 149.461 ms 171.109 ms</span></div>
<div class="p1">
<span class="s1">17 * * *</span></div>
<br />
<div class="p1">
<span class="s1">18 217.191-100-28.etapanet.net (191.100.28.217) 178.335 ms * *</span></div>
<br />
<div class="p1">
<span class="s1">[root@nuc12-i7 ~]# firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='188.40.95.70' reject"</span></div>
<br />
<div class="p1">
<span class="s1">[root@nuc12-i7 ~]# firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='191.100.28.217' reject"</span></div>
</div>
</div>
</div>
</div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-38626184602639535222015-09-13T12:51:00.000-07:002015-09-14T04:30:43.292-07:00Spring Externalized Configuration Design PatternsThis post is a dump of details surrounding setting up a configuration ecosystem for WAR deployments that is independent and as stateless as possible.<br />
Some of these patterns I learned while developing the Drug Database and API for Telus Health. However, the major portion of these patterns I picked up while developing with ProntoForms Inc.<br />
<br />
<h2>
Requirements</h2>
<br />
R1) Environment variance: Some of the configuration will vary depending on whether we are in a production, staging, qa or dev deployment.<br />
R2) Developer and Task specific: We will allow for developer and task discriminators to further specialize and override properties<br />
R3) Plural properties: We need the ability to specify arbitrary sets or lists of property values - via XML or YAML<br />
<br />
<br />
<br />
<h2>
</h2>
<h2>
Implementation</h2>
Normally I would put the ecosystem, discriminator and developer variables in the @Value annotation to do some runtime dynamic substitution. However, this would target only a specific node in the properties tree. What we need is a layered resolution of properties where we override any properties previously loaded with those deeper in the tree. We accomplish this by loading the properties in sequence.<br />
<br />
<div class="p1">
<div class="p1">
<context:property-placeholder<span class="s1"> </span></div>
<div class="p2">
<span class="s1"> </span><span class="s2">location</span><span class="s3">=</span>"file:///${os.environment.configuration.dir}//${os.environment.ecosystem}/biometric.properties"<span class="s1"> </span></div>
<div class="p3">
<span class="s1"> </span>order<span class="s3">=</span><span class="s4">"1"</span><span class="s1"> </span>ignore-unresolvable<span class="s3">=</span><span class="s4">"true"</span><span class="s1"> </span>ignore-resource-not-found<span class="s3">=</span><span class="s4">"true"</span><span class="s3">/></span></div>
<div class="p1">
<context:property-placeholder<span class="s1"> </span></div>
<div class="p2">
<span class="s1"> </span><span class="s2">location</span><span class="s3">=</span>"file:///${os.environment.configuration.dir}//${os.environment.ecosystem}/${os.environment.discriminator:}/biometric.properties"<span class="s1"> </span></div>
<div class="p3">
<span class="s1"> </span>order<span class="s3">=</span><span class="s4">"2"</span><span class="s1"> </span>ignore-unresolvable<span class="s3">=</span><span class="s4">"true"</span><span class="s1"> </span>ignore-resource-not-found<span class="s3">=</span><span class="s4">"true"</span><span class="s3">/></span></div>
<div class="p1">
<context:property-placeholder<span class="s1"> </span></div>
<div class="p2">
<span class="s1"> </span><span class="s2">location</span><span class="s3">=</span>"file:///${os.environment.configuration.dir}//${os.environment.ecosystem}/${os.environment.developer.username:}/biometric.properties"<span class="s1"> </span></div>
<div class="p3">
<span class="s1"> </span>order<span class="s3">=</span><span class="s4">"3"</span><span class="s1"> </span>ignore-unresolvable<span class="s3">=</span><span class="s4">"true"</span><span class="s1"> </span>ignore-resource-not-found<span class="s3">=</span><span class="s4">"true"</span><span class="s3">/></span></div>
<div class="p1">
<context:property-placeholder<span class="s1"> </span></div>
<div class="p2">
<span class="s1"> </span><span class="s2">location</span><span class="s3">=</span>"file:///${os.environment.configuration.dir}//${os.environment.ecosystem}/${os.environment.discriminator:}/${os.environment.developer.username:}/biometric.properties"<span class="s1"> </span></div>
<br />
<div class="p3">
<span class="s1"> </span>order<span class="s3">=</span><span class="s4">"4"</span><span class="s1"> </span>ignore-unresolvable<span class="s3">=</span><span class="s4">"true"</span><span class="s1"> </span>ignore-resource-not-found<span class="s3">=</span><span class="s4">"true"</span><span class="s3">/></span></div>
</div>
<br />
<div class="p4">
</div>
<h3>
System environment variables</h3>
Tomcat container - Launch Configuration | Environment<br />
os.environment.ecosystem=dev<br />
os.environment.developer=obrien<br />
os.environment.discriminator=<empty><br />
os.environment.configuration.dir=/wsc/os/config<br />
<br />
<h3>
File System property files</h3>
/wsc/os/config/biometric.properties<br />
os.environment.persistencecontext.applicationservice.name=from<br />
<br />
Note: for external resources you also need the following for missing directories<br />
<br />
<div class="p1">
<span class="s1">ignore-resource-not-found=</span><span class="s2">"true"</span></div>
<h3>
Spring context xml</h3>
<div>
<div class="p1">
<context:property-placeholder<span class="s1"> </span></div>
<div class="p2">
<span class="s1"> </span><span class="s2">location</span><span class="s3">=</span>"file:///${os.environment.configuration.dir}/biometric.properties"<span class="s1"> </span><span class="s3">/></span></div>
<div class="p2">
<span class="s3"><br /></span></div>
<div class="p2">
<span class="s3">
</span></div>
<div class="p1">
<util:list<span class="s1"> </span><span class="s2">id</span>=<span class="s3">"locationsList"</span>></div>
<div class="p1">
<value>${os.environment.configuration.dir:classpath:}/${os.environment.ecosystem}}</value></div>
<div class="p1">
<value>${os.environment.configuration.dir:classpath:}/${os.environment.ecosystem}/${os.environment.discriminator:}}</value></div>
<div class="p1">
<value>${os.environment.configuration.dir:classpath:}/${os.environment.ecosystem}/${os.environment.developer.username:}}</value></div>
<div class="p1">
<value>${os.environment.configuration.dir:classpath:}/${os.environment.ecosystem}/${os.environment.discriminator:}/${os.environment.developer.username:}}</value></div>
<div class="p1">
</div>
<div class="p1">
</util:list></div>
<div class="p1">
<br /></div>
</div>
<h3>
Spring Bean</h3>
<br />
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span></span><span class="s2"><b>private</b></span><span class="s1"> </span><span class="s3"><b><i>@Value</i></b></span><span class="s4">(</span><span class="s5">"${server}"</span><span class="s4">)</span><span class="s1"> </span><b>String</b><span class="s1"> </span>server<span class="s4">;</span></div>
<div class="p1">
<span class="s3"><b><i>@PersistenceContext</i></b></span><span class="s4">(</span><span class="s7">name</span><span class="s4">=</span><span class="s5">"${os.environment.persistencecontext.applicationservice.name}"</span><span class="s4">)</span></div>
<br />
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span></span><span class="s2"><b>private</b></span><span class="s1"> </span>EntityManager<span class="s1"> </span>entityManager<span class="s4">;</span><br />
<span class="s4"><br /></span>
<span class="s4"><br /></span>
<span class="s4">References</span><br />
<span class="s4"><a href="https://jira.spring.io/browse/SPR-5719">https://jira.spring.io/browse/SPR-5719</a></span><br />
<a href="https://jira.spring.io/browse/SPR-6428">https://jira.spring.io/browse/SPR-6428</a><br />
<br /></div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-29978763744714451882015-06-28T08:47:00.000-07:002016-11-12T06:14:34.038-08:00OpenStack Private CloudSetup<br />
Ubuntu MaaS and DevStack are free, but Redhat requires a subscription per/node - buy the $99 developer server license at <a href="https://www.redhat.com/apps/store/developers/">https://www.redhat.com/apps/store/developers/</a><br />
<br />
Redhat Director (RHEL OpenStack - Kilo)<br />
<a href="https://access.redhat.com/products/red-hat-enterprise-linux-openstack-platform">https://access.redhat.com/products/red-hat-enterprise-linux-openstack-platform</a><br />
<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/7/html/Director_Installation_and_Usage/index.html">https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/7/html/Director_Installation_and_Usage/index.html</a><br />
<br />
Get Redhat Enterprise Linux 7<br />
<a href="https://access.redhat.com/products/red-hat-enterprise-linux-openstack-platform/evaluation">https://access.redhat.com/products/red-hat-enterprise-linux-openstack-platform/evaluation</a><br />
or Ubunto (<a href="http://docs.openstack.org/image-guide/content/ubuntu-image.html">http://docs.openstack.org/image-guide/content/ubuntu-image.html</a>)<br />
I am using RHEL<br />
Install to at least 3 boxes<br />
Enable networking (nmcli d, nmtui)<br />
<br />
Checkout RDO <a href="https://www.rdoproject.org/Quickstart">https://www.rdoproject.org/Quickstart</a><br />
<a href="https://www.youtube.com/watch?v=DGf-ny25OAw">https://www.youtube.com/watch?v=DGf-ny25OAw</a><br />
<br />
Set proxy in ~/.bashrc and<br />
/etc/yum.conf<br />
proxy=http://global.proxy<br />
<br />
verify /etc/yum.repos.d/redhat.repo and rdo.release.repo<br />
<br />
(behind a proxy the following does not work) - use a cellular connection at work temporarily<br />
<pre class="highlight plaintext" style="background: rgb(238, 238, 238); border-radius: 4px; border: none; box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 20px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 10px 15px; word-break: break-all; word-wrap: normal;"><span style="white-space: pre-wrap;">verify /etc/hosts has the correct ip set for the VM</span></pre>
<pre class="highlight plaintext" style="background: rgb(238, 238, 238); border-radius: 4px; border: none; box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 20px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 10px 15px; word-break: break-all; word-wrap: normal;"><code style="background-color: transparent; border-radius: 0px; box-sizing: border-box; color: inherit; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: inherit; padding: 0px; white-space: pre-wrap;">sudo yum update -y
sudo yum install -y https://www.rdoproject.org/repos/rdo-release.rpm
sudo vi /etc/yum/pluginconf.d/search-disabled-repos.conf
sudo yum install -y openstack-packstack
packstack --allinone <span style="background-color: #ecf0f3; color: #222222; font-family: monospace; font-size: 12.87px; line-height: 21.6px; white-space: normal;">--os-heat-install=y</span>
cat keystonerc_admin
openstack-status
</code></pre>
<pre class="highlight plaintext" style="background: rgb(238, 238, 238); border-radius: 4px; border: none; box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 20px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 10px 15px; word-break: break-all; word-wrap: normal;"><code style="background-color: transparent; border-radius: 0px; box-sizing: border-box; color: inherit; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: inherit; padding: 0px; white-space: pre-wrap;">http://192.168.0.113/dashboard</code></pre>
Register System<br />
Attach Subscription<br />
sudo yum update -y<br />
sudo vi /etc/yum/pluginconf.d/search-disabled-repos.conf<br />
'notify_only=0'<br />
sudo yum install -y http://rdo.fedorapeople.org/rdo-release.rpm<br />
<br />
to solve<br />
--> Finished Dependency Resolution<br />
Error: Package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch (openstack-liberty)<br />
Requires: python-docutils<br />
**********************************************************************<br />
yum can be configured to try to resolve such errors by temporarily enabling<br />
disabled repos and searching for missing dependencies.<br />
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf<br />
**********************************************************************<br />
<br />
Error: Package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch (openstack-liberty)<br />
Requires: python-docutils<br />
You could try using --skip-broken to work around the problem<br />
You could try running: rpm -Va --nofiles --nodigest<br />
<br />
[openstack@biomos30 ~]$ sudo vi /etc/yum/pluginconf.d/search-disabled-repos.conf<br />
[<br />
<br />
sudo vi <span style="background-color: white; font-family: monospace; font-size: 16px; line-height: 20.8px;">/etc/sysconfig/network-scripts/ifcfg-*</span><br />
<br />
<br />
# dhcp<br />
#BOOTPROTO=dhcp<br />
# static<br />
BOOTPROTO=none<br />
NETWORK=192.168.0.0<br />
NETMASK=255.255.255.0<br />
IPADDR=192.168.0.101<br />
USERCTL=no<br />
<br />
<br />
<br />
# dhcp<br />
#BOOTPROTO=dhcp<br />
# static<br />
BOOTPROTO=static<br />
NETWORK=192.168.2.0<br />
NETMASK=255.255.255.0<br />
IPADDR=192.168.2.101<br />
USERCTL=no<br />
DNS1=192.168.2.1<br />
GATEWAY=192.168.2.1<br />
<br />
sudo ip link set wlp7s0u1 up<br />
<br />
[root@nuc0 ~(keystone_admin)]# keystone service-list<br />
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient.<br />
'python-keystoneclient.', DeprecationWarning)<br />
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning: Constructing an instance of the keystoneclient.v2_0.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.<br />
'the 2.0.0 release.', DeprecationWarning)<br />
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead<br />
super(Client, self).__init__(**kwargs)<br />
/usr/lib/python2.7/site-packages/debtcollector/renames.py:43: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead<br />
return f(*args, **kwargs)<br />
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:376: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.<br />
'the 2.0.0 release.', DeprecationWarning)<br />
+----------------------------------+------------+--------------+--------------------------------+<br />
| id | name | type | description |<br />
+----------------------------------+------------+--------------+--------------------------------+<br />
| 2d09a8805e084dab9d6b410ac2f0bde2 | ceilometer | metering | Openstack Metering Service |<br />
| cddf5dfa558e4a82b36f4b7e4b9a31ad | cinder | volume | Cinder Service |<br />
| 6eaa647639834c14a4b5f5c567d133e2 | cinderv2 | volumev2 | Cinder Service v2 |<br />
| 33e1e03563954fc59f6689a9dbedafbc | glance | image | OpenStack Image Service |<br />
| f381a2299abb475b80e47cbb58748ed4 | keystone | identity | OpenStack Identity Service |<br />
| a77128c659714b11bb3b39d6ec30f840 | neutron | network | Neutron Networking Service |<br />
| 0c0ea0c00a1348b293310f621bd7aefa | nova | compute | Openstack Compute Service |<br />
| f66e375e60c949359a29fb3c50083072 | nova_ec2 | ec2 | EC2 Service |<br />
| b1a186652fae4f71a984a19cb092aa48 | novav3 | computev3 | Openstack Compute Service v3 |<br />
| e65fac36e71e498088625118606e390b | swift | object-store | Openstack Object-Store Service |<br />
| fe5206ce62d042a280be017e97648a6f | swift_s3 | s3 | Openstack S3 Service |<br />
+----------------------------------+------------+--------------+--------------------------------+<br />
<div>
<br />
[openstack@nuc0 ~]$ source keystonerc_admin<br />
[openstack@nuc0 ~(keystone_admin)]$ neutron net-list<br />
+--------------------------------------+---------+------------------------------------------------------+<br />
| id | name | subnets |<br />
+--------------------------------------+---------+------------------------------------------------------+<br />
| df58b3b2-c236-485b-8e56-3974304aaf2f | public | 47395dc1-6dc1-46d5-86e1-43c6c5c0cc00 172.24.4.224/28 |<br />
| 89827ddf-7a4f-4cd9-88ee-b1df6f712059 | private | e7f9c637-d34e-4bb9-8641-67b86c0b48e4 10.0.0.0/24 |<br />
+--------------------------------------+---------+------------------------------------------------------+</div>
<div>
<br /></div>
<br />
--------------------------------------------------------------------------------------------------------------------------------------------------------<br />
Total 1.1 MB/s | 84 MB 00:01:18 <br />
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release<br />
Importing GPG key 0xFD431D51:<br />
Userid : "Red Hat, Inc. (release key 2) <security@redhat.com>"<br />
Fingerprint: 567e 347a d004 4ade 55ba 8a5f 199e 2f91 fd43 1d51<br />
Package : redhat-release-server-7.2-9.el7.x86_64 (@anaconda/7.2)<br />
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release<br />
Importing GPG key 0x2FA658E0:<br />
Userid : "Red Hat, Inc. (auxiliary key) <security@redhat.com>"<br />
Fingerprint: 43a6 e49c 4a38 f4be 9abf 2a53 4568 9c88 2fa6 58e0<br />
Package : redhat-release-server-7.2-9.el7.x86_64 (@anaconda/7.2)<br />
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release<br />
Running transaction check<br />
Running transaction test<br />
Transaction test succeeded<br />
Running transaction<br />
Updating : glibc-2.17-106.el7_2.1.x86_64 1/151 <br />
Updating : glibc-common-2.17-106.el7_2.1.x86_64 2/151 <br />
Updating : libxml2-2.9.1-6.el7_2.2.x86_64 3/151 <br />
Updating : dracut-033-360.el7_2.x86_64 4/151 <br />
Updating : 1:openssl-libs-1.0.1e-51.el7_2.1.x86_64 5/151 <br />
Updating : 10:qemu-img-1.5.3-105.el7_2.1.x86_64 6/151 <br />
Updating : 32:bind-license-9.9.4-29.el7_2.1.noarch 7/151 <br />
Updating : 32:bind-libs-9.9.4-29.el7_2.1.x86_64 8/151 <br />
Updating : libvirt-client-1.2.17-13.el7_2.2.x86_64 9/151 <br />
Updating : libvirt-daemon-1.2.17-13.el7_2.2.x86_64 10/151 <br />
Updating : libvirt-daemon-driver-network-1.2.17-13.el7_2.2.x86_64 11/151 <br />
Updating : libvirt-daemon-driver-qemu-1.2.17-13.el7_2.2.x86_64 12/151 <br />
Updating : libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.2.x86_64 13/151 <br />
Updating : libvirt-daemon-driver-storage-1.2.17-13.el7_2.2.x86_64 14/151 <br />
Updating : libvirt-daemon-driver-secret-1.2.17-13.el7_2.2.x86_64 15/151 <br />
Updating : libvirt-daemon-driver-interface-1.2.17-13.el7_2.2.x86_64 16/151 <br />
Updating : libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.2.x86_64 17/151 <br />
Updating : 10:qemu-kvm-common-1.5.3-105.el7_2.1.x86_64 18/151 <br />
Updating : 2:libpng-1.5.13-7.el7_2.x86_64 19/151 <br />
Updating : 10:qemu-kvm-1.5.3-105.el7_2.1.x86_64 20/151 <br />
Updating : libsss_idmap-1.13.0-40.el7_2.1.x86_64 21/151 <br />
Updating : kernel-tools-libs-3.10.0-327.3.1.el7.x86_64 22/151 <br />
Updating : 1:gmp-6.0.0-12.el7_1.x86_64 23/151 <br />
Updating : 1:grub2-tools-2.02-0.33.el7_2.x86_64 24/151 <br />
Updating : libsss_nss_idmap-1.13.0-40.el7_2.1.x86_64 25/151 <br />
Updating : python-perf-3.10.0-327.3.1.el7.x86_64 26/151 <br />
Updating : libreport-filesystem-2.1.11-31.el7.x86_64 27/151 <br />
Updating : libreport-python-2.1.11-31.el7.x86_64 28/151 <br />
Updating : libreport-2.1.11-31.el7.x86_64 29/151 <br />
Updating : abrt-libs-2.1.11-35.el7.x86_64 30/151 <br />
Updating : libreport-web-2.1.11-31.el7.x86_64 31/151 <br />
Updating : libreport-plugin-rhtsupport-2.1.11-31.el7.x86_64 32/151 <br />
Updating : libreport-plugin-bugzilla-2.1.11-31.el7.x86_64 33/151 <br />
Updating : libreport-plugin-reportuploader-2.1.11-31.el7.x86_64 34/151 <br />
Updating : libreport-rhel-2.1.11-31.el7.x86_64 35/151 <br />
Updating : libreport-gtk-2.1.11-31.el7.x86_64 36/151 <br />
Updating : libreport-plugin-ureport-2.1.11-31.el7.x86_64 37/151 <br />
Updating : abrt-python-2.1.11-35.el7.x86_64 38/151 <br />
Updating : abrt-2.1.11-35.el7.x86_64 39/151 <br />
Updating : abrt-dbus-2.1.11-35.el7.x86_64 40/151 <br />
Updating : abrt-addon-kerneloops-2.1.11-35.el7.x86_64 41/151 <br />
Updating : abrt-addon-pstoreoops-2.1.11-35.el7.x86_64 42/151 <br />
Updating : abrt-addon-vmcore-2.1.11-35.el7.x86_64 43/151 <br />
Updating : abrt-addon-python-2.1.11-35.el7.x86_64 44/151 <br />
Updating : abrt-addon-ccpp-2.1.11-35.el7.x86_64 45/151 <br />
Updating : abrt-addon-xorg-2.1.11-35.el7.x86_64 46/151 <br />
Updating : abrt-gui-libs-2.1.11-35.el7.x86_64 47/151 <br />
Updating : abrt-gui-2.1.11-35.el7.x86_64 48/151 <br />
Updating : libreport-cli-2.1.11-31.el7.x86_64 49/151 <br />
Updating : abrt-tui-2.1.11-35.el7.x86_64 50/151 <br />
Updating : abrt-cli-2.1.11-35.el7.x86_64 51/151 <br />
Updating : abrt-console-notification-2.1.11-35.el7.x86_64 52/151 <br />
Updating : abrt-desktop-2.1.11-35.el7.x86_64 53/151 <br />
Updating : libreport-anaconda-2.1.11-31.el7.x86_64 54/151 <br />
Updating : libreport-rhel-anaconda-bugzilla-2.1.11-31.el7.x86_64 55/151 <br />
Updating : libreport-plugin-mailx-2.1.11-31.el7.x86_64 56/151 <br />
Updating : tuned-2.5.1-4.el7_2.1.noarch 57/151 <br />
Updating : sssd-client-1.13.0-40.el7_2.1.x86_64 58/151 <br />
Updating : 1:grub2-efi-2.02-0.33.el7_2.x86_64 59/151 <br />
Updating : libreswan-3.15-5.el7_1.x86_64 60/151 <br />
Updating : kernel-tools-3.10.0-327.3.1.el7.x86_64 61/151 <br />
Updating : libvirt-daemon-kvm-1.2.17-13.el7_2.2.x86_64 62/151 <br />
Updating : libvirt-daemon-config-network-1.2.17-13.el7_2.2.x86_64 63/151 <br />
Updating : 32:bind-utils-9.9.4-29.el7_2.1.x86_64 64/151 <br />
Updating : 32:bind-libs-lite-9.9.4-29.el7_2.1.x86_64 65/151 <br />
Updating : 1:openssl-1.0.1e-51.el7_2.1.x86_64 66/151 <br />
Installing : kernel-3.10.0-327.3.1.el7.x86_64 67/151 <br />
Updating : dracut-config-rescue-033-360.el7_2.x86_64 68/151 <br />
Updating : dracut-network-033-360.el7_2.x86_64 69/151 <br />
Updating : rdma-7.2_4.1_rc6-2.el7.noarch 70/151 <br />
Updating : libxml2-python-2.9.1-6.el7_2.2.x86_64 71/151 <br />
Updating : crash-7.1.2-3.el7_2.x86_64 72/151 <br />
Updating : logrotate-3.8.6-7.el7_2.x86_64 73/151 <br />
Updating : 10:libcacard-1.5.3-105.el7_2.1.x86_64 74/151 <br />
Updating : firefox-38.5.0-3.el7_2.x86_64 75/151 <br />
Updating : anaconda-user-help-19.31.123-1.el7.x86_64 76/151 <br />
Cleanup : abrt-desktop-2.1.11-34.el7.x86_64 77/151 <br />
Cleanup : libvirt-daemon-kvm-1.2.17-13.el7.x86_64 78/151 <br />
Cleanup : abrt-gui-2.1.11-34.el7.x86_64 79/151 <br />
Cleanup : 32:bind-utils-9.9.4-29.el7.x86_64 80/151 <br />
Cleanup : 32:bind-libs-9.9.4-29.el7.x86_64 81/151 <br />
Cleanup : 32:bind-libs-lite-9.9.4-29.el7.x86_64 82/151 <br />
Cleanup : libvirt-daemon-driver-qemu-1.2.17-13.el7.x86_64 83/151 <br />
Cleanup : 10:qemu-kvm-1.5.3-105.el7.x86_64 84/151 <br />
Cleanup : libreport-gtk-2.1.11-30.el7.x86_64 85/151 <br />
Cleanup : libvirt-daemon-driver-storage-1.2.17-13.el7.x86_64 86/151 <br />
Cleanup : 1:openssl-1.0.1e-42.el7_1.9.x86_64 87/151 <br />
Cleanup : libreport-anaconda-2.1.11-30.el7.x86_64 88/151 <br />
Cleanup : libreport-plugin-reportuploader-2.1.11-30.el7.x86_64 89/151 <br />
Cleanup : kernel-tools-3.10.0-327.el7.x86_64 90/151 <br />
Cleanup : sssd-client-1.13.0-40.el7.x86_64 91/151 <br />
Cleanup : abrt-gui-libs-2.1.11-34.el7.x86_64 92/151 <br />
Cleanup : libvirt-daemon-driver-interface-1.2.17-13.el7.x86_64 93/151 <br />
Cleanup : libvirt-daemon-driver-nodedev-1.2.17-13.el7.x86_64 94/151 <br />
Cleanup : libvirt-daemon-driver-nwfilter-1.2.17-13.el7.x86_64 95/151 <br />
Cleanup : libvirt-daemon-driver-secret-1.2.17-13.el7.x86_64 96/151 <br />
Cleanup : libreport-plugin-mailx-2.1.11-30.el7.x86_64 97/151 <br />
Cleanup : libreswan-3.12-10.1.el7_1.x86_64 98/151 <br />
Cleanup : libxml2-python-2.9.1-5.el7_1.2.x86_64 99/151 <br />
Cleanup : abrt-console-notification-2.1.11-34.el7.x86_64 100/151 <br />
Cleanup : abrt-cli-2.1.11-34.el7.x86_64 101/151 <br />
Cleanup : abrt-addon-vmcore-2.1.11-34.el7.x86_64 102/151 <br />
Cleanup : libvirt-daemon-config-network-1.2.17-13.el7.x86_64 103/151 <br />
Cleanup : libreport-rhel-anaconda-bugzilla-2.1.11-30.el7.x86_64 104/151 <br />
Cleanup : libreport-rhel-2.1.11-30.el7.x86_64 105/151 <br />
Cleanup : rdma-7.2_4.1_rc6-1.el7.noarch 106/151 <br />
Cleanup : firefox-38.3.0-2.el7_1.x86_64 107/151 <br />
Cleanup : 1:grub2-efi-2.02-0.29.el7.x86_64 108/151 <br />
Cleanup : tuned-2.5.1-4.el7.noarch 109/151 <br />
Cleanup : dracut-network-033-359.el7.x86_64 110/151 <br />
Cleanup : dracut-config-rescue-033-359.el7.x86_64 111/151 <br />
Cleanup : 32:bind-license-9.9.4-29.el7.noarch 112/151 <br />
Cleanup : anaconda-user-help-7.2.2-1.el7.noarch 113/151 <br />
Cleanup : abrt-tui-2.1.11-34.el7.x86_64 114/151 <br />
Cleanup : libreport-plugin-bugzilla-2.1.11-30.el7.x86_64 115/151 <br />
Cleanup : abrt-addon-pstoreoops-2.1.11-34.el7.x86_64 116/151 <br />
Cleanup : abrt-addon-ccpp-2.1.11-34.el7.x86_64 117/151 <br />
Cleanup : abrt-addon-kerneloops-2.1.11-34.el7.x86_64 118/151 <br />
Cleanup : abrt-addon-python-2.1.11-34.el7.x86_64 119/151 <br />
Cleanup : abrt-addon-xorg-2.1.11-34.el7.x86_64 120/151 <br />
Cleanup : abrt-python-2.1.11-34.el7.x86_64 121/151 <br />
Cleanup : abrt-dbus-2.1.11-34.el7.x86_64 122/151 <br />
Cleanup : abrt-2.1.11-34.el7.x86_64 123/151 <br />
Cleanup : libreport-plugin-rhtsupport-2.1.11-30.el7.x86_64 124/151 <br />
Cleanup : libreport-plugin-ureport-2.1.11-30.el7.x86_64 125/151 <br />
Cleanup : libreport-web-2.1.11-30.el7.x86_64 126/151 <br />
Cleanup : libvirt-daemon-driver-network-1.2.17-13.el7.x86_64 127/151 <br />
Cleanup : libvirt-daemon-1.2.17-13.el7.x86_64 128/151 <br />
Cleanup : libvirt-client-1.2.17-13.el7.x86_64 129/151 <br />
Cleanup : abrt-libs-2.1.11-34.el7.x86_64 130/151 <br />
Cleanup : libreport-cli-2.1.11-30.el7.x86_64 131/151 <br />
Cleanup : libreport-python-2.1.11-30.el7.x86_64 132/151 <br />
Cleanup : libreport-2.1.11-30.el7.x86_64 133/151 <br />
Cleanup : 10:qemu-kvm-common-1.5.3-105.el7.x86_64 134/151 <br />
Cleanup : libxml2-2.9.1-5.el7_1.2.x86_64 135/151 <br />
Cleanup : dracut-033-359.el7.x86_64 136/151 <br />
Cleanup : python-perf-3.10.0-327.el7.x86_64 137/151 <br />
Cleanup : 1:grub2-tools-2.02-0.29.el7.x86_64 138/151 <br />
Cleanup : 1:gmp-6.0.0-11.el7.x86_64 139/151 <br />
Cleanup : libsss_idmap-1.13.0-40.el7.x86_64 140/151 <br />
Cleanup : libsss_nss_idmap-1.13.0-40.el7.x86_64 141/151 <br />
Cleanup : kernel-tools-libs-3.10.0-327.el7.x86_64 142/151 <br />
Cleanup : 1:openssl-libs-1.0.1e-42.el7_1.9.x86_64 143/151 <br />
Cleanup : 10:qemu-img-1.5.3-105.el7.x86_64 144/151 <br />
Cleanup : 2:libpng-1.5.13-5.el7.x86_64 145/151 <br />
Cleanup : 10:libcacard-1.5.3-105.el7.x86_64 146/151 <br />
Cleanup : logrotate-3.8.6-6.el7.x86_64 147/151 <br />
Cleanup : crash-7.1.2-2.el7.x86_64 148/151 <br />
Cleanup : libreport-filesystem-2.1.11-30.el7.x86_64 149/151 <br />
Cleanup : glibc-common-2.17-105.el7.x86_64 150/151 <br />
Cleanup : glibc-2.17-105.el7.x86_64 151/151 <br />
rhel-7-server-eus-rpms/7Server/x86_64/productid | 1.7 kB 00:00:00 <br />
rhel-7-server-rpms/7Server/x86_64/productid | 1.7 kB 00:00:00 <br />
rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/productid | 1.8 kB 00:00:00 <br />
rhel-ha-for-rhel-7-server-rpms/7Server/x86_64/productid | 1.7 kB 00:00:00 <br />
rhel-rs-for-rhel-7-server-eus-rpms/7Server/x86_64/productid | 1.8 kB 00:00:00 <br />
rhel-rs-for-rhel-7-server-rpms/7Server/x86_64/productid | 1.7 kB 00:00:00 <br />
Verifying : 2:libpng-1.5.13-7.el7_2.x86_64 1/151 <br />
Verifying : abrt-desktop-2.1.11-35.el7.x86_64 2/151 <br />
Verifying : libxml2-python-2.9.1-6.el7_2.2.x86_64 3/151 <br />
Verifying : libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.2.x86_64 4/151 <br />
Verifying : abrt-addon-python-2.1.11-35.el7.x86_64 5/151 <br />
Verifying : libreswan-3.15-5.el7_1.x86_64 6/151 <br />
Verifying : libvirt-daemon-driver-storage-1.2.17-13.el7_2.2.x86_64 7/151 <br />
Verifying : kernel-3.10.0-327.3.1.el7.x86_64 8/151 <br />
Verifying : libreport-rhel-2.1.11-31.el7.x86_64 9/151 <br />
Verifying : abrt-addon-pstoreoops-2.1.11-35.el7.x86_64 10/151 <br />
Verifying : abrt-2.1.11-35.el7.x86_64 11/151 <br />
Verifying : 1:openssl-1.0.1e-51.el7_2.1.x86_64 12/151 <br />
Verifying : libreport-plugin-bugzilla-2.1.11-31.el7.x86_64 13/151 <br />
Verifying : libreport-plugin-rhtsupport-2.1.11-31.el7.x86_64 14/151 <br />
Verifying : abrt-addon-ccpp-2.1.11-35.el7.x86_64 15/151 <br />
Verifying : 10:qemu-kvm-common-1.5.3-105.el7_2.1.x86_64 16/151 <br />
Verifying : anaconda-user-help-19.31.123-1.el7.x86_64 17/151 <br />
Verifying : libreport-filesystem-2.1.11-31.el7.x86_64 18/151 <br />
Verifying : libvirt-daemon-driver-secret-1.2.17-13.el7_2.2.x86_64 19/151 <br />
Verifying : abrt-gui-2.1.11-35.el7.x86_64 20/151 <br />
Verifying : abrt-gui-libs-2.1.11-35.el7.x86_64 21/151 <br />
Verifying : glibc-common-2.17-106.el7_2.1.x86_64 22/151 <br />
Verifying : crash-7.1.2-3.el7_2.x86_64 23/151 <br />
Verifying : libreport-rhel-anaconda-bugzilla-2.1.11-31.el7.x86_64 24/151 <br />
Verifying : libsss_idmap-1.13.0-40.el7_2.1.x86_64 25/151 <br />
Verifying : sssd-client-1.13.0-40.el7_2.1.x86_64 26/151 <br />
Verifying : dracut-config-rescue-033-360.el7_2.x86_64 27/151 <br />
Verifying : kernel-tools-libs-3.10.0-327.3.1.el7.x86_64 28/151 <br />
Verifying : 10:qemu-kvm-1.5.3-105.el7_2.1.x86_64 29/151 <br />
Verifying : dracut-network-033-360.el7_2.x86_64 30/151 <br />
Verifying : libvirt-daemon-driver-interface-1.2.17-13.el7_2.2.x86_64 31/151 <br />
Verifying : abrt-libs-2.1.11-35.el7.x86_64 32/151 <br />
Verifying : abrt-cli-2.1.11-35.el7.x86_64 33/151 <br />
Verifying : libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.2.x86_64 34/151 <br />
Verifying : 1:gmp-6.0.0-12.el7_1.x86_64 35/151 <br />
Verifying : libreport-plugin-ureport-2.1.11-31.el7.x86_64 36/151 <br />
Verifying : abrt-tui-2.1.11-35.el7.x86_64 37/151 <br />
Verifying : libvirt-daemon-config-network-1.2.17-13.el7_2.2.x86_64 38/151 <br />
Verifying : libreport-plugin-mailx-2.1.11-31.el7.x86_64 39/151 <br />
Verifying : libvirt-daemon-kvm-1.2.17-13.el7_2.2.x86_64 40/151 <br />
Verifying : libreport-cli-2.1.11-31.el7.x86_64 41/151 <br />
Verifying : 1:grub2-tools-2.02-0.33.el7_2.x86_64 42/151 <br />
Verifying : tuned-2.5.1-4.el7_2.1.noarch 43/151 <br />
Verifying : libreport-2.1.11-31.el7.x86_64 44/151 <br />
Verifying : 32:bind-libs-9.9.4-29.el7_2.1.x86_64 45/151 <br />
Verifying : logrotate-3.8.6-7.el7_2.x86_64 46/151 <br />
Verifying : glibc-2.17-106.el7_2.1.x86_64 47/151 <br />
Verifying : libvirt-daemon-driver-qemu-1.2.17-13.el7_2.2.x86_64 48/151 <br />
Verifying : libreport-web-2.1.11-31.el7.x86_64 49/151 <br />
Verifying : 1:grub2-efi-2.02-0.33.el7_2.x86_64 50/151 <br />
Verifying : abrt-addon-xorg-2.1.11-35.el7.x86_64 51/151 <br />
Verifying : libxml2-2.9.1-6.el7_2.2.x86_64 52/151 <br />
Verifying : abrt-addon-vmcore-2.1.11-35.el7.x86_64 53/151 <br />
Verifying : libreport-plugin-reportuploader-2.1.11-31.el7.x86_64 54/151 <br />
Verifying : 10:libcacard-1.5.3-105.el7_2.1.x86_64 55/151 <br />
Verifying : 10:qemu-img-1.5.3-105.el7_2.1.x86_64 56/151 <br />
Verifying : abrt-addon-kerneloops-2.1.11-35.el7.x86_64 57/151 <br />
Verifying : libvirt-daemon-driver-network-1.2.17-13.el7_2.2.x86_64 58/151 <br />
Verifying : libreport-anaconda-2.1.11-31.el7.x86_64 59/151 <br />
Verifying : 32:bind-license-9.9.4-29.el7_2.1.noarch 60/151 <br />
Verifying : libreport-gtk-2.1.11-31.el7.x86_64 61/151 <br />
Verifying : abrt-python-2.1.11-35.el7.x86_64 62/151 <br />
Verifying : abrt-console-notification-2.1.11-35.el7.x86_64 63/151 <br />
Verifying : firefox-38.5.0-3.el7_2.x86_64 64/151 <br />
Verifying : kernel-tools-3.10.0-327.3.1.el7.x86_64 65/151 <br />
Verifying : libsss_nss_idmap-1.13.0-40.el7_2.1.x86_64 66/151 <br />
Verifying : 32:bind-libs-lite-9.9.4-29.el7_2.1.x86_64 67/151 <br />
Verifying : 32:bind-utils-9.9.4-29.el7_2.1.x86_64 68/151 <br />
Verifying : libvirt-client-1.2.17-13.el7_2.2.x86_64 69/151 <br />
Verifying : rdma-7.2_4.1_rc6-2.el7.noarch 70/151 <br />
Verifying : libreport-python-2.1.11-31.el7.x86_64 71/151 <br />
Verifying : libvirt-daemon-1.2.17-13.el7_2.2.x86_64 72/151 <br />
Verifying : dracut-033-360.el7_2.x86_64 73/151 <br />
Verifying : 1:openssl-libs-1.0.1e-51.el7_2.1.x86_64 74/151 <br />
Verifying : abrt-dbus-2.1.11-35.el7.x86_64 75/151 <br />
Verifying : python-perf-3.10.0-327.3.1.el7.x86_64 76/151 <br />
Verifying : dracut-config-rescue-033-359.el7.x86_64 77/151 <br />
Verifying : 10:qemu-img-1.5.3-105.el7.x86_64 78/151 <br />
Verifying : abrt-addon-python-2.1.11-34.el7.x86_64 79/151 <br />
Verifying : abrt-tui-2.1.11-34.el7.x86_64 80/151 <br />
Verifying : glibc-common-2.17-105.el7.x86_64 81/151 <br />
Verifying : libvirt-daemon-driver-nodedev-1.2.17-13.el7.x86_64 82/151 <br />
Verifying : libvirt-daemon-driver-nwfilter-1.2.17-13.el7.x86_64 83/151 <br />
Verifying : 1:openssl-libs-1.0.1e-42.el7_1.9.x86_64 84/151 <br />
Verifying : 32:bind-license-9.9.4-29.el7.noarch 85/151 <br />
Verifying : libvirt-daemon-driver-qemu-1.2.17-13.el7.x86_64 86/151 <br />
Verifying : abrt-desktop-2.1.11-34.el7.x86_64 87/151 <br />
Verifying : libvirt-daemon-driver-secret-1.2.17-13.el7.x86_64 88/151 <br />
Verifying : abrt-cli-2.1.11-34.el7.x86_64 89/151 <br />
Verifying : abrt-gui-2.1.11-34.el7.x86_64 90/151 <br />
Verifying : abrt-gui-libs-2.1.11-34.el7.x86_64 91/151 <br />
Verifying : libreport-plugin-ureport-2.1.11-30.el7.x86_64 92/151 <br />
Verifying : libreswan-3.12-10.1.el7_1.x86_64 93/151 <br />
Verifying : libreport-plugin-rhtsupport-2.1.11-30.el7.x86_64 94/151 <br />
Verifying : libvirt-daemon-driver-storage-1.2.17-13.el7.x86_64 95/151 <br />
Verifying : sssd-client-1.13.0-40.el7.x86_64 96/151 <br />
Verifying : rdma-7.2_4.1_rc6-1.el7.noarch 97/151 <br />
Verifying : 32:bind-libs-lite-9.9.4-29.el7.x86_64 98/151 <br />
Verifying : abrt-addon-ccpp-2.1.11-34.el7.x86_64 99/151 <br />
Verifying : 32:bind-utils-9.9.4-29.el7.x86_64 100/151 <br />
Verifying : libvirt-daemon-config-network-1.2.17-13.el7.x86_64 101/151 <br />
Verifying : kernel-tools-3.10.0-327.el7.x86_64 102/151 <br />
Verifying : 1:openssl-1.0.1e-42.el7_1.9.x86_64 103/151 <br />
Verifying : 2:libpng-1.5.13-5.el7.x86_64 104/151 <br />
Verifying : abrt-addon-vmcore-2.1.11-34.el7.x86_64 105/151 <br />
Verifying : libreport-rhel-anaconda-bugzilla-2.1.11-30.el7.x86_64 106/151 <br />
Verifying : libreport-plugin-reportuploader-2.1.11-30.el7.x86_64 107/151 <br />
Verifying : 1:gmp-6.0.0-11.el7.x86_64 108/151 <br />
Verifying : libvirt-daemon-driver-interface-1.2.17-13.el7.x86_64 109/151 <br />
Verifying : kernel-tools-libs-3.10.0-327.el7.x86_64 110/151 <br />
Verifying : glibc-2.17-105.el7.x86_64 111/151 <br />
Verifying : libreport-2.1.11-30.el7.x86_64 112/151 <br />
Verifying : libreport-cli-2.1.11-30.el7.x86_64 113/151 <br />
Verifying : libreport-plugin-bugzilla-2.1.11-30.el7.x86_64 114/151 <br />
Verifying : abrt-console-notification-2.1.11-34.el7.x86_64 115/151 <br />
Verifying : 1:grub2-efi-2.02-0.29.el7.x86_64 116/151 <br />
Verifying : dracut-network-033-359.el7.x86_64 117/151 <br />
Verifying : abrt-2.1.11-34.el7.x86_64 118/151 <br />
Verifying : abrt-addon-xorg-2.1.11-34.el7.x86_64 119/151 <br />
Verifying : abrt-dbus-2.1.11-34.el7.x86_64 120/151 <br />
Verifying : logrotate-3.8.6-6.el7.x86_64 121/151 <br />
Verifying : libreport-python-2.1.11-30.el7.x86_64 122/151 <br />
Verifying : anaconda-user-help-7.2.2-1.el7.noarch 123/151 <br />
Verifying : 10:libcacard-1.5.3-105.el7.x86_64 124/151 <br />
Verifying : abrt-addon-kerneloops-2.1.11-34.el7.x86_64 125/151 <br />
Verifying : abrt-libs-2.1.11-34.el7.x86_64 126/151 <br />
Verifying : libsss_nss_idmap-1.13.0-40.el7.x86_64 127/151 <br />
Verifying : libvirt-daemon-driver-network-1.2.17-13.el7.x86_64 128/151 <br />
Verifying : firefox-38.3.0-2.el7_1.x86_64 129/151 <br />
Verifying : 10:qemu-kvm-common-1.5.3-105.el7.x86_64 130/151 <br />
Verifying : tuned-2.5.1-4.el7.noarch 131/151 <br />
Verifying : libxml2-python-2.9.1-5.el7_1.2.x86_64 132/151 <br />
Verifying : 10:qemu-kvm-1.5.3-105.el7.x86_64 133/151 <br />
Verifying : libreport-web-2.1.11-30.el7.x86_64 134/151 <br />
Verifying : libreport-plugin-mailx-2.1.11-30.el7.x86_64 135/151 <br />
Verifying : dracut-033-359.el7.x86_64 136/151 <br />
Verifying : libreport-filesystem-2.1.11-30.el7.x86_64 137/151 <br />
Verifying : abrt-addon-pstoreoops-2.1.11-34.el7.x86_64 138/151 <br />
Verifying : libreport-gtk-2.1.11-30.el7.x86_64 139/151 <br />
Verifying : libvirt-daemon-1.2.17-13.el7.x86_64 140/151 <br />
Verifying : crash-7.1.2-2.el7.x86_64 141/151 <br />
Verifying : libreport-rhel-2.1.11-30.el7.x86_64 142/151 <br />
Verifying : libvirt-client-1.2.17-13.el7.x86_64 143/151 <br />
Verifying : 1:grub2-tools-2.02-0.29.el7.x86_64 144/151 <br />
Verifying : libsss_idmap-1.13.0-40.el7.x86_64 145/151 <br />
Verifying : libxml2-2.9.1-5.el7_1.2.x86_64 146/151 <br />
Verifying : 32:bind-libs-9.9.4-29.el7.x86_64 147/151 <br />
Verifying : libvirt-daemon-kvm-1.2.17-13.el7.x86_64 148/151 <br />
Verifying : abrt-python-2.1.11-34.el7.x86_64 149/151 <br />
Verifying : python-perf-3.10.0-327.el7.x86_64 150/151 <br />
Verifying : libreport-anaconda-2.1.11-30.el7.x86_64 151/151 <br />
<br />
Installed:<br />
kernel.x86_64 0:3.10.0-327.3.1.el7 <br />
<br />
Updated:<br />
abrt.x86_64 0:2.1.11-35.el7 abrt-addon-ccpp.x86_64 0:2.1.11-35.el7 <br />
abrt-addon-kerneloops.x86_64 0:2.1.11-35.el7 abrt-addon-pstoreoops.x86_64 0:2.1.11-35.el7 <br />
abrt-addon-python.x86_64 0:2.1.11-35.el7 abrt-addon-vmcore.x86_64 0:2.1.11-35.el7 <br />
abrt-addon-xorg.x86_64 0:2.1.11-35.el7 abrt-cli.x86_64 0:2.1.11-35.el7 <br />
abrt-console-notification.x86_64 0:2.1.11-35.el7 abrt-dbus.x86_64 0:2.1.11-35.el7 <br />
abrt-desktop.x86_64 0:2.1.11-35.el7 abrt-gui.x86_64 0:2.1.11-35.el7 <br />
abrt-gui-libs.x86_64 0:2.1.11-35.el7 abrt-libs.x86_64 0:2.1.11-35.el7 <br />
abrt-python.x86_64 0:2.1.11-35.el7 abrt-tui.x86_64 0:2.1.11-35.el7 <br />
anaconda-user-help.x86_64 0:19.31.123-1.el7 bind-libs.x86_64 32:9.9.4-29.el7_2.1 <br />
bind-libs-lite.x86_64 32:9.9.4-29.el7_2.1 bind-license.noarch 32:9.9.4-29.el7_2.1 <br />
bind-utils.x86_64 32:9.9.4-29.el7_2.1 crash.x86_64 0:7.1.2-3.el7_2 <br />
dracut.x86_64 0:033-360.el7_2 dracut-config-rescue.x86_64 0:033-360.el7_2 <br />
dracut-network.x86_64 0:033-360.el7_2 firefox.x86_64 0:38.5.0-3.el7_2 <br />
glibc.x86_64 0:2.17-106.el7_2.1 glibc-common.x86_64 0:2.17-106.el7_2.1 <br />
gmp.x86_64 1:6.0.0-12.el7_1 grub2-efi.x86_64 1:2.02-0.33.el7_2 <br />
grub2-tools.x86_64 1:2.02-0.33.el7_2 kernel-tools.x86_64 0:3.10.0-327.3.1.el7 <br />
kernel-tools-libs.x86_64 0:3.10.0-327.3.1.el7 libcacard.x86_64 10:1.5.3-105.el7_2.1 <br />
libpng.x86_64 2:1.5.13-7.el7_2 libreport.x86_64 0:2.1.11-31.el7 <br />
libreport-anaconda.x86_64 0:2.1.11-31.el7 libreport-cli.x86_64 0:2.1.11-31.el7 <br />
libreport-filesystem.x86_64 0:2.1.11-31.el7 libreport-gtk.x86_64 0:2.1.11-31.el7 <br />
libreport-plugin-bugzilla.x86_64 0:2.1.11-31.el7 libreport-plugin-mailx.x86_64 0:2.1.11-31.el7 <br />
libreport-plugin-reportuploader.x86_64 0:2.1.11-31.el7 libreport-plugin-rhtsupport.x86_64 0:2.1.11-31.el7 <br />
libreport-plugin-ureport.x86_64 0:2.1.11-31.el7 libreport-python.x86_64 0:2.1.11-31.el7 <br />
libreport-rhel.x86_64 0:2.1.11-31.el7 libreport-rhel-anaconda-bugzilla.x86_64 0:2.1.11-31.el7 <br />
libreport-web.x86_64 0:2.1.11-31.el7 libreswan.x86_64 0:3.15-5.el7_1 <br />
libsss_idmap.x86_64 0:1.13.0-40.el7_2.1 libsss_nss_idmap.x86_64 0:1.13.0-40.el7_2.1 <br />
libvirt-client.x86_64 0:1.2.17-13.el7_2.2 libvirt-daemon.x86_64 0:1.2.17-13.el7_2.2 <br />
libvirt-daemon-config-network.x86_64 0:1.2.17-13.el7_2.2 libvirt-daemon-driver-interface.x86_64 0:1.2.17-13.el7_2.2 <br />
libvirt-daemon-driver-network.x86_64 0:1.2.17-13.el7_2.2 libvirt-daemon-driver-nodedev.x86_64 0:1.2.17-13.el7_2.2 <br />
libvirt-daemon-driver-nwfilter.x86_64 0:1.2.17-13.el7_2.2 libvirt-daemon-driver-qemu.x86_64 0:1.2.17-13.el7_2.2 <br />
libvirt-daemon-driver-secret.x86_64 0:1.2.17-13.el7_2.2 libvirt-daemon-driver-storage.x86_64 0:1.2.17-13.el7_2.2 <br />
libvirt-daemon-kvm.x86_64 0:1.2.17-13.el7_2.2 libxml2.x86_64 0:2.9.1-6.el7_2.2 <br />
libxml2-python.x86_64 0:2.9.1-6.el7_2.2 logrotate.x86_64 0:3.8.6-7.el7_2 <br />
openssl.x86_64 1:1.0.1e-51.el7_2.1 openssl-libs.x86_64 1:1.0.1e-51.el7_2.1 <br />
python-perf.x86_64 0:3.10.0-327.3.1.el7 qemu-img.x86_64 10:1.5.3-105.el7_2.1 <br />
qemu-kvm.x86_64 10:1.5.3-105.el7_2.1 qemu-kvm-common.x86_64 10:1.5.3-105.el7_2.1 <br />
rdma.noarch 0:7.2_4.1_rc6-2.el7 sssd-client.x86_64 0:1.13.0-40.el7_2.1 <br />
tuned.noarch 0:2.5.1-4.el7_2.1 <br />
<br />
Complete!<br />
[openstack@nuc0 ~]$ sudo yum install -y http://rdo.fedorapeople.org/rdo-release.rpm<br />
[sudo] password for openstack: <br />
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager<br />
rdo-release.rpm | 5.1 kB 00:00:00 <br />
Examining /var/tmp/yum-root-DJeZGF/rdo-release.rpm: rdo-release-liberty-2.noarch<br />
Marking /var/tmp/yum-root-DJeZGF/rdo-release.rpm to be installed<br />
Resolving Dependencies<br />
--> Running transaction check<br />
---> Package rdo-release.noarch 0:liberty-2 will be installed<br />
--> Finished Dependency Resolution<br />
<br />
Dependencies Resolved<br />
<br />
========================================================================================================================================================<br />
Package Arch Version Repository Size<br />
========================================================================================================================================================<br />
Installing:<br />
rdo-release noarch liberty-2 /rdo-release 1.4 k<br />
<br />
Transaction Summary<br />
========================================================================================================================================================<br />
Install 1 Package<br />
<br />
Total size: 1.4 k<br />
Installed size: 1.4 k<br />
Downloading packages:<br />
Running transaction check<br />
Running transaction test<br />
Transaction test succeeded<br />
Running transaction<br />
Installing : rdo-release-liberty-2.noarch 1/1 <br />
Verifying : rdo-release-liberty-2.noarch 1/1 <br />
<br />
Installed:<br />
rdo-release.noarch 0:liberty-2 <br />
<br />
Complete!<br />
[openstack@nuc0 ~]$ sudo packstack --allinone<br />
[sudo] password for openstack: <br />
sudo: packstack: command not found<br />
[openstack@nuc0 ~]$ sudo yum install -y openstack-packstack<br />
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager<br />
openstack-liberty | 2.9 kB 00:00:00 <br />
openstack-liberty/x86_64/primary_db | 389 kB 00:00:00 <br />
Resolving Dependencies<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: openstack-packstack-puppet = 1:7.0.0-0.7.dev1661.gaf13b7e.el7 for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: openstack-puppet-modules >= 2014.2.10 for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: python-netaddr for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: PyYAML for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Running transaction check<br />
---> Package PyYAML.x86_64 0:3.10-11.el7 will be installed<br />
--> Processing Dependency: libyaml-0.so.2()(64bit) for package: PyYAML-3.10-11.el7.x86_64<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package openstack-packstack-puppet.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
---> Package openstack-puppet-modules.noarch 1:7.0.0-1.el7 will be installed<br />
--> Processing Dependency: rubygem-json for package: 1:openstack-puppet-modules-7.0.0-1.el7.noarch<br />
---> Package python-netaddr.noarch 0:0.7.18-1.el7 will be installed<br />
--> Running transaction check<br />
---> Package libyaml.x86_64 0:0.1.4-11.el7_0 will be installed<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package rubygem-json.x86_64 0:1.7.7-25.el7_1 will be installed<br />
--> Processing Dependency: ruby(rubygems) >= 2.0.14 for package: rubygem-json-1.7.7-25.el7_1.x86_64<br />
--> Processing Dependency: ruby(release) for package: rubygem-json-1.7.7-25.el7_1.x86_64<br />
--> Processing Dependency: libruby.so.2.0()(64bit) for package: rubygem-json-1.7.7-25.el7_1.x86_64<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package ruby-libs.x86_64 0:2.0.0.598-25.el7_1 will be installed<br />
---> Package rubygems.noarch 0:2.0.14-25.el7_1 will be installed<br />
--> Processing Dependency: rubygem(rdoc) >= 4.0.0 for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Processing Dependency: rubygem(psych) >= 2.0.0 for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Processing Dependency: rubygem(io-console) >= 0.4.2 for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Processing Dependency: /usr/bin/ruby for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package ruby.x86_64 0:2.0.0.598-25.el7_1 will be installed<br />
--> Processing Dependency: rubygem(bigdecimal) >= 1.2.0 for package: ruby-2.0.0.598-25.el7_1.x86_64<br />
---> Package rubygem-io-console.x86_64 0:0.4.2-25.el7_1 will be installed<br />
---> Package rubygem-psych.x86_64 0:2.0.0-25.el7_1 will be installed<br />
---> Package rubygem-rdoc.noarch 0:4.0.0-25.el7_1 will be installed<br />
--> Processing Dependency: ruby(irb) = 2.0.0.598 for package: rubygem-rdoc-4.0.0-25.el7_1.noarch<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package ruby-irb.noarch 0:2.0.0.598-25.el7_1 will be installed<br />
---> Package rubygem-bigdecimal.x86_64 0:1.2.0-25.el7_1 will be installed<br />
--> Finished Dependency Resolution<br />
Error: Package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch (openstack-liberty)<br />
Requires: python-docutils<br />
**********************************************************************<br />
yum can be configured to try to resolve such errors by temporarily enabling<br />
disabled repos and searching for missing dependencies.<br />
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf<br />
**********************************************************************<br />
<br />
Error: Package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch (openstack-liberty)<br />
Requires: python-docutils<br />
You could try using --skip-broken to work around the problem<br />
You could try running: rpm -Va --nofiles --nodigest<br />
[openstack@nuc0 ~]$ sudo vi /etc/yum/pluginconf.d/search-disabled-repos.conf <br />
[openstack@nuc0 ~]$ sudo yum install -y openstack-packstack<br />
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager<br />
Resolving Dependencies<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: openstack-packstack-puppet = 1:7.0.0-0.7.dev1661.gaf13b7e.el7 for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: openstack-puppet-modules >= 2014.2.10 for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: python-netaddr for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Processing Dependency: PyYAML for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Running transaction check<br />
---> Package PyYAML.x86_64 0:3.10-11.el7 will be installed<br />
--> Processing Dependency: libyaml-0.so.2()(64bit) for package: PyYAML-3.10-11.el7.x86_64<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package openstack-packstack-puppet.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
---> Package openstack-puppet-modules.noarch 1:7.0.0-1.el7 will be installed<br />
--> Processing Dependency: rubygem-json for package: 1:openstack-puppet-modules-7.0.0-1.el7.noarch<br />
---> Package python-netaddr.noarch 0:0.7.18-1.el7 will be installed<br />
--> Running transaction check<br />
---> Package libyaml.x86_64 0:0.1.4-11.el7_0 will be installed<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package rubygem-json.x86_64 0:1.7.7-25.el7_1 will be installed<br />
--> Processing Dependency: ruby(rubygems) >= 2.0.14 for package: rubygem-json-1.7.7-25.el7_1.x86_64<br />
--> Processing Dependency: ruby(release) for package: rubygem-json-1.7.7-25.el7_1.x86_64<br />
--> Processing Dependency: libruby.so.2.0()(64bit) for package: rubygem-json-1.7.7-25.el7_1.x86_64<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package ruby-libs.x86_64 0:2.0.0.598-25.el7_1 will be installed<br />
---> Package rubygems.noarch 0:2.0.14-25.el7_1 will be installed<br />
--> Processing Dependency: rubygem(rdoc) >= 4.0.0 for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Processing Dependency: rubygem(psych) >= 2.0.0 for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Processing Dependency: rubygem(io-console) >= 0.4.2 for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Processing Dependency: /usr/bin/ruby for package: rubygems-2.0.14-25.el7_1.noarch<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package ruby.x86_64 0:2.0.0.598-25.el7_1 will be installed<br />
--> Processing Dependency: rubygem(bigdecimal) >= 1.2.0 for package: ruby-2.0.0.598-25.el7_1.x86_64<br />
---> Package rubygem-io-console.x86_64 0:0.4.2-25.el7_1 will be installed<br />
---> Package rubygem-psych.x86_64 0:2.0.0-25.el7_1 will be installed<br />
---> Package rubygem-rdoc.noarch 0:4.0.0-25.el7_1 will be installed<br />
--> Processing Dependency: ruby(irb) = 2.0.0.598 for package: rubygem-rdoc-4.0.0-25.el7_1.noarch<br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
---> Package ruby-irb.noarch 0:2.0.0.598-25.el7_1 will be installed<br />
---> Package rubygem-bigdecimal.x86_64 0:1.2.0-25.el7_1 will be installed<br />
--> Finished Dependency Resolution<br />
Error: Package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch (openstack-liberty)<br />
Requires: python-docutils<br />
**********************************************************************<br />
Dependency resolving failed due to missing dependencies.<br />
Some repositories on your system are disabled, but yum can enable them<br />
and search for missing dependencies. This will require downloading<br />
metadata for disabled repositories and may take some time and traffic.<br />
**********************************************************************<br />
<br />
rhel-7-server-eus-rhn-tools-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-eus-rhn-tools-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-eus-rhn-tools-rpms/7Server/x86_64/updateinfo | 24 kB 00:00:00 <br />
rhel-7-server-eus-rhn-tools-rpms/7Server/x86_64/primary_db | 31 kB 00:00:00 <br />
rhel-7-server-openstack-7.0-tools-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-openstack-7.0-tools-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-openstack-7.0-tools-rpms/7Server/x86_64/updateinfo | 76 kB 00:00:00 <br />
rhel-7-server-openstack-7.0-tools-rpms/7Server/x86_64/primary_db | 32 kB 00:00:00 <br />
rhel-7-server-eus-rh-common-rpms/7Server/x86_64 | 3.7 kB 00:00:00 <br />
rhel-7-server-eus-rh-common-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-eus-rh-common-rpms/7Server/x86_64/updateinfo | 91 B 00:00:00 <br />
rhel-7-server-eus-rh-common-rpms/7Server/x86_64/primary_db | 1.5 kB 00:00:00 <br />
rhel-7-server-eus-satellite-tools-6.1-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-eus-satellite-tools-6.1-rpms/7Server/x86_64/group | 1.7 kB 00:00:00 <br />
rhel-7-server-eus-satellite-tools-6.1-rpms/7Server/x86_64/updateinfo | 34 kB 00:00:00 <br />
rhel-7-server-eus-satellite-tools-6.1-rpms/7Server/x86_64/primary_db | 34 kB 00:00:00 <br />
rhel-ha-for-rhel-7-server-fastrack-rpms/x86_64 | 2.5 kB 00:00:00 <br />
rhel-ha-for-rhel-7-server-fastrack-rpms/x86_64/primary_db | 1.2 kB 00:00:00 <br />
rhel-7-server-v2vwin-1-rpms/7Server/x86_64 | 2.9 kB 00:00:00 <br />
rhel-7-server-v2vwin-1-rpms/7Server/x86_64/updateinfo | 1.1 kB 00:00:00 <br />
rhel-7-server-v2vwin-1-rpms/7Server/x86_64/primary_db | 3.5 kB 00:00:00 <br />
rh-gluster-3-client-for-rhel-7-server-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rh-gluster-3-client-for-rhel-7-server-rpms/7Server/x86_64/group | 130 B 00:00:00 <br />
rh-gluster-3-client-for-rhel-7-server-rpms/7Server/x86_64/updateinfo | 7.7 kB 00:00:00 <br />
rh-gluster-3-client-for-rhel-7-server-rpms/7Server/x86_64/primary_db | 15 kB 00:00:00 <br />
rhel-7-server-extras-rpms/x86_64 | 2.9 kB 00:00:00 <br />
rhel-7-server-extras-rpms/x86_64/updateinfo | 35 kB 00:00:00 <br />
rhel-7-server-extras-rpms/x86_64/primary_db | 68 kB 00:00:00 <br />
rhel-7-server-eus-supplementary-rpms/7Server/x86_64 | 2.9 kB 00:00:00 <br />
rhel-7-server-eus-supplementary-rpms/7Server/x86_64/updateinfo | 5.3 kB 00:00:00 <br />
rhel-7-server-eus-supplementary-rpms/7Server/x86_64/primary_db | 80 kB 00:00:00 <br />
rhel-server-rhscl-7-eus-rpms/7Server/x86_64 | 2.9 kB 00:00:00 <br />
https://cdn.redhat.com/content/eus/rhel/server/7/7Server/x86_64/rhscl/1/os/repodata/8ef08056443a777124f17d328ae519fdc98fdd72343bb70a7172656795c7176d-updateinfo.xml.gz: [Errno 14] HTTPS Error 404 - Not Found<br />
Trying other mirror.<br />
To address this issue please refer to the below knowledge base article <br />
<br />
https://access.redhat.com/articles/1320623<br />
<br />
If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.<br />
<br />
rhel-server-rhscl-7-eus-rpms/7 FAILED <br />
https://cdn.redhat.com/content/eus/rhel/server/7/7Server/x86_64/rhscl/1/os/repodata/75b08da25f22440831380ad08414ad4954fe4823-primary.sqlite.bz2: [Errno 14] HTTPS Error 404 - Not Found<br />
Trying other mirror.<br />
https://cdn.redhat.com/content/eus/rhel/server/7/7Server/x86_64/rhscl/1/os/repodata/75b08da25f22440831380ad08414ad4954fe4823-primary.sqlite.bz2: [Errno 14] HTTPS Error 404 - Not Found<br />
Trying other mirror.<br />
rhel-server-rhscl-7-rpms/7Server/x86_64 | 3.1 kB 00:00:00 <br />
rhel-server-rhscl-7-rpms/7Server/x86_64/updateinfo | 363 kB 00:00:00 <br />
rhel-server-rhscl-7-rpms/7Server/x86_64/primary_db | 1.8 MB 00:00:00 <br />
rhel-7-server-eus-optional-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-eus-optional-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-eus-optional-rpms/7Server/x86_64/updateinfo | 560 kB 00:00:00 <br />
rhel-7-server-eus-optional-rpms/7Server/x86_64/primary_db | 3.1 MB 00:00:01 <br />
rhel-7-server-rhn-tools-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-rhn-tools-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-rhn-tools-rpms/7Server/x86_64/updateinfo | 24 kB 00:00:00 <br />
rhel-7-server-rhn-tools-rpms/7Server/x86_64/primary_db | 30 kB 00:00:00 <br />
rhel-rs-for-rhel-7-server-fastrack-rpms/x86_64 | 2.5 kB 00:00:00 <br />
rhel-rs-for-rhel-7-server-fastrack-rpms/x86_64/primary_db | 1.2 kB 00:00:00 <br />
rhel-7-server-thirdparty-oracle-java-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-thirdparty-oracle-java-rpms/7Server/x86_64/group_gz | 5.2 kB 00:00:00 <br />
rhel-7-server-thirdparty-oracle-java-rpms/7Server/x86_64/updateinfo | 16 kB 00:00:00 <br />
rhel-7-server-thirdparty-oracle-java-rpms/7Server/x86_64/primary_db | 159 kB 00:00:00 <br />
rhel-7-server-fastrack-rpms/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-fastrack-rpms/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-fastrack-rpms/x86_64/updateinfo | 31 kB 00:00:00 <br />
rhel-7-server-fastrack-rpms/x86_64/primary_db | 153 kB 00:00:00 <br />
rhel-7-server-rh-common-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-rh-common-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-rh-common-rpms/7Server/x86_64/updateinfo | 47 kB 00:00:00 <br />
rhel-7-server-rh-common-rpms/7Server/x86_64/primary_db | 89 kB 00:00:00 <br />
rhel-7-server-optional-fastrack-rpms/x86_64 | 2.9 kB 00:00:00 <br />
rhel-7-server-optional-fastrack-rpms/x86_64/updateinfo | 8.2 kB 00:00:00 <br />
rhel-7-server-optional-fastrack-rpms/x86_64/primary_db | 51 kB 00:00:00 <br />
rhel-7-server-satellite-tools-6.1-rpms/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-satellite-tools-6.1-rpms/x86_64/group | 1.7 kB 00:00:00 <br />
rhel-7-server-satellite-tools-6.1-rpms/x86_64/updateinfo | 34 kB 00:00:00 <br />
rhel-7-server-satellite-tools-6.1-rpms/x86_64/primary_db | 34 kB 00:00:00 <br />
rhel-atomic-host-rpms/x86_64 | 4.0 kB 00:00:00 <br />
rhel-atomic-host-rpms/x86_64/group | 104 B 00:00:00 <br />
rhel-atomic-host-rpms/x86_64/updateinfo | 15 kB 00:00:00 <br />
rhel-atomic-host-rpms/x86_64/primary_db | 56 kB 00:00:00 <br />
rhel-7-server-supplementary-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-supplementary-rpms/7Server/x86_64/group_gz | 8.9 kB 00:00:00 <br />
rhel-7-server-supplementary-rpms/7Server/x86_64/updateinfo | 19 kB 00:00:00 <br />
rhel-7-server-supplementary-rpms/7Server/x86_64/primary_db | 106 kB 00:00:00 <br />
rhel-7-server-optional-rpms/7Server/x86_64 | 3.5 kB 00:00:00 <br />
rhel-7-server-optional-rpms/7Server/x86_64/group_gz | 6.2 kB 00:00:00 <br />
rhel-7-server-optional-rpms/7Server/x86_64/updateinfo | 792 kB 00:00:00 <br />
rhel-7-server-optional-rpms/7Server/x86_64/primary_db | 3.0 MB 00:00:01 <br />
rhel-7-server-eus-supplementary-debuginfo/7Server/x86_64 | 2.5 kB 00:00:00 <br />
rhel-7-server-eus-supplementary-debuginfo/7Server/x86_64/primary_db | 1.2 kB 00:00:00 <br />
rhel-7-server-rhceph-1.3-tools-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
rhel-7-server-rhceph-1.3-tools-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-7-server-rhceph-1.3-tools-rpms/7Server/x86_64/updateinfo | 35 kB 00:00:00 <br />
rhel-7-server-rhceph-1.3-tools-rpms/7Server/x86_64/primary_db | 46 kB 00:00:00 <br />
rhel-7-server-eus-thirdparty-oracle-java-rpms/7Server/x86_64 | 3.8 kB 00:00:00 <br />
https://cdn.redhat.com/content/eus/rhel/server/7/7Server/x86_64/oracle-java/os/repodata/3e800f8eb4bfc89f8e320cda3dd709a8e0bef5f4884e3b622df63b17249e8242-comps.xml: [Errno 14] HTTPS Error 404 - Not Found<br />
Trying other mirror.<br />
rhel-7-server-eus-thirdparty-o FAILED <br />
https://cdn.redhat.com/content/eus/rhel/server/7/7Server/x86_64/oracle-java/os/repodata/c6ae45356b83c5a8997be192c69cd945ea3ac6f991a7eaba6c90a7f17d0728a7-primary.sqlite.bz2: [Errno 14] HTTPS Error 404 - Not Found<br />
Trying other mirror.<br />
https://cdn.redhat.com/content/eus/rhel/server/7/7Server/x86_64/oracle-java/os/repodata/c6ae45356b83c5a8997be192c69cd945ea3ac6f991a7eaba6c90a7f17d0728a7-primary.sqlite.bz2: [Errno 14] HTTPS Error 404 - Not Found<br />
Trying other mirror.<br />
rhel-sjis-for-rhel-7-server-eus-rpms/7Server/x86_64 | 4.0 kB 00:00:00 <br />
rhel-sjis-for-rhel-7-server-eus-rpms/7Server/x86_64/group | 104 B 00:00:00 <br />
rhel-sjis-for-rhel-7-server-eus-rpms/7Server/x86_64/updateinfo | 91 B 00:00:00 <br />
rhel-sjis-for-rhel-7-server-eus-rpms/7Server/x86_64/primary_db | 1.5 kB 00:00:00 <br />
--> Running transaction check<br />
---> Package openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 will be installed<br />
--> Processing Dependency: python-docutils for package: 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch<br />
--> Running transaction check<br />
---> Package python-docutils.noarch 0:0.11-0.2.20130715svn7687.el7 will be installed<br />
--> Processing Dependency: python-imaging for package: python-docutils-0.11-0.2.20130715svn7687.el7.noarch<br />
--> Running transaction check<br />
---> Package python-pillow.x86_64 0:2.0.0-19.gitd1c6db8.el7 will be installed<br />
--> Finished Dependency Resolution<br />
<br />
Dependencies Resolved<br />
<br />
========================================================================================================================================================<br />
Package Arch Version Repository Size<br />
========================================================================================================================================================<br />
Installing:<br />
openstack-packstack noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 openstack-liberty 232 k<br />
Installing for dependencies:<br />
PyYAML x86_64 3.10-11.el7 rhel-7-server-eus-rpms 153 k<br />
libyaml x86_64 0.1.4-11.el7_0 rhel-7-server-eus-rpms 55 k<br />
openstack-packstack-puppet noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 openstack-liberty 17 k<br />
openstack-puppet-modules noarch 1:7.0.0-1.el7 openstack-liberty 1.7 M<br />
python-docutils noarch 0.11-0.2.20130715svn7687.el7 rhel-7-server-eus-optional-rpms 1.5 M<br />
python-netaddr noarch 0.7.18-1.el7 openstack-liberty 1.3 M<br />
python-pillow x86_64 2.0.0-19.gitd1c6db8.el7 rhel-7-server-eus-rpms 438 k<br />
ruby x86_64 2.0.0.598-25.el7_1 rhel-7-server-eus-rpms 67 k<br />
ruby-irb noarch 2.0.0.598-25.el7_1 rhel-7-server-eus-rpms 88 k<br />
ruby-libs x86_64 2.0.0.598-25.el7_1 rhel-7-server-eus-rpms 2.8 M<br />
rubygem-bigdecimal x86_64 1.2.0-25.el7_1 rhel-7-server-eus-rpms 79 k<br />
rubygem-io-console x86_64 0.4.2-25.el7_1 rhel-7-server-eus-rpms 50 k<br />
rubygem-json x86_64 1.7.7-25.el7_1 rhel-7-server-eus-rpms 75 k<br />
rubygem-psych x86_64 2.0.0-25.el7_1 rhel-7-server-eus-rpms 77 k<br />
rubygem-rdoc noarch 4.0.0-25.el7_1 rhel-7-server-eus-rpms 318 k<br />
rubygems noarch 2.0.14-25.el7_1 rhel-7-server-eus-rpms 212 k<br />
<br />
Transaction Summary<br />
========================================================================================================================================================<br />
Install 1 Package (+16 Dependent packages)<br />
<br />
Total download size: 9.1 M<br />
Installed size: 33 M<br />
Downloading packages:<br />
(1/17): libyaml-0.1.4-11.el7_0.x86_64.rpm | 55 kB 00:00:00 <br />
warning: /var/cache/yum/x86_64/7Server/openstack-liberty/packages/openstack-packstack-puppet-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch.rpm: Header V4 RSA/SHA1 Signature, key ID 764429e6: NOKEY<br />
Public key for openstack-packstack-puppet-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch.rpm is not installed<br />
(2/17): openstack-packstack-puppet-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch.rpm | 17 kB 00:00:00 <br />
(3/17): openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch.rpm | 232 kB 00:00:00 <br />
(4/17): PyYAML-3.10-11.el7.x86_64.rpm | 153 kB 00:00:00 <br />
(5/17): ruby-2.0.0.598-25.el7_1.x86_64.rpm | 67 kB 00:00:00 <br />
(6/17): python-pillow-2.0.0-19.gitd1c6db8.el7.x86_64.rpm | 438 kB 00:00:00 <br />
(7/17): ruby-irb-2.0.0.598-25.el7_1.noarch.rpm | 88 kB 00:00:00 <br />
(8/17): rubygem-bigdecimal-1.2.0-25.el7_1.x86_64.rpm | 79 kB 00:00:00 <br />
(9/17): rubygem-io-console-0.4.2-25.el7_1.x86_64.rpm | 50 kB 00:00:00 <br />
(10/17): rubygem-json-1.7.7-25.el7_1.x86_64.rpm | 75 kB 00:00:00 <br />
(11/17): rubygem-psych-2.0.0-25.el7_1.x86_64.rpm | 77 kB 00:00:00 <br />
(12/17): python-netaddr-0.7.18-1.el7.noarch.rpm | 1.3 MB 00:00:02 <br />
(13/17): python-docutils-0.11-0.2.20130715svn7687.el7.noarch.rpm | 1.5 MB 00:00:02 <br />
(14/17): openstack-puppet-modules-7.0.0-1.el7.noarch.rpm | 1.7 MB 00:00:03 <br />
(15/17): rubygem-rdoc-4.0.0-25.el7_1.noarch.rpm | 318 kB 00:00:00 <br />
(16/17): rubygems-2.0.14-25.el7_1.noarch.rpm | 212 kB 00:00:00 <br />
(17/17): ruby-libs-2.0.0.598-25.el7_1.x86_64.rpm | 2.8 MB 00:00:03 <br />
--------------------------------------------------------------------------------------------------------------------------------------------------------<br />
Total 2.0 MB/s | 9.1 MB 00:00:04 <br />
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud<br />
Importing GPG key 0x764429E6:<br />
Userid : "CentOS Cloud SIG (http://wiki.centos.org/SpecialInterestGroup/Cloud) <security@centos.org>"<br />
Fingerprint: 736a f511 6d9c 40e2 af6b 074b f9b9 fee7 7644 29e6<br />
Package : rdo-release-liberty-2.noarch (@/rdo-release)<br />
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud<br />
Running transaction check<br />
Running transaction test<br />
Transaction test succeeded<br />
Running transaction<br />
Installing : ruby-libs-2.0.0.598-25.el7_1.x86_64 1/17 <br />
Installing : libyaml-0.1.4-11.el7_0.x86_64 2/17 <br />
Installing : PyYAML-3.10-11.el7.x86_64 3/17 <br />
Installing : ruby-irb-2.0.0.598-25.el7_1.noarch 4/17 <br />
Installing : ruby-2.0.0.598-25.el7_1.x86_64 5/17 <br />
Installing : rubygem-bigdecimal-1.2.0-25.el7_1.x86_64 6/17 <br />
Installing : rubygem-rdoc-4.0.0-25.el7_1.noarch 7/17 <br />
Installing : rubygem-json-1.7.7-25.el7_1.x86_64 8/17 <br />
Installing : rubygem-io-console-0.4.2-25.el7_1.x86_64 9/17 <br />
Installing : rubygems-2.0.14-25.el7_1.noarch 10/17 <br />
Installing : rubygem-psych-2.0.0-25.el7_1.x86_64 11/17 <br />
Installing : 1:openstack-puppet-modules-7.0.0-1.el7.noarch 12/17 <br />
Installing : 1:openstack-packstack-puppet-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch 13/17 <br />
Installing : python-netaddr-0.7.18-1.el7.noarch 14/17 <br />
Installing : python-pillow-2.0.0-19.gitd1c6db8.el7.x86_64 15/17 <br />
Installing : python-docutils-0.11-0.2.20130715svn7687.el7.noarch 16/17 <br />
Installing : 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch 17/17 <br />
rhel-7-server-supplementary-rpms/7Server/x86_64/productid | 1.6 kB 00:00:00 <br />
rhel-7-server-thirdparty-oracle-java-rpms/7Server/x86_64/productid | 1.7 kB 00:00:00 <br />
rhel-atomic-host-rpms/x86_64/productid | 2.1 kB 00:00:00 <br />
rhel-server-rhscl-7-rpms/7Server/x86_64/productid | 1.7 kB 00:00:00 <br />
rhel-sjis-for-rhel-7-server-eus-rpms/7Server/x86_64/productid | 2.2 kB 00:00:00 <br />
Verifying : libyaml-0.1.4-11.el7_0.x86_64 1/17 <br />
Verifying : python-pillow-2.0.0-19.gitd1c6db8.el7.x86_64 2/17 <br />
Verifying : rubygem-rdoc-4.0.0-25.el7_1.noarch 3/17 <br />
Verifying : rubygems-2.0.14-25.el7_1.noarch 4/17 <br />
Verifying : 1:openstack-packstack-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch 5/17 <br />
Verifying : python-netaddr-0.7.18-1.el7.noarch 6/17 <br />
Verifying : ruby-2.0.0.598-25.el7_1.x86_64 7/17 <br />
Verifying : rubygem-bigdecimal-1.2.0-25.el7_1.x86_64 8/17 <br />
Verifying : 1:openstack-puppet-modules-7.0.0-1.el7.noarch 9/17 <br />
Verifying : ruby-libs-2.0.0.598-25.el7_1.x86_64 10/17 <br />
Verifying : rubygem-json-1.7.7-25.el7_1.x86_64 11/17 <br />
Verifying : PyYAML-3.10-11.el7.x86_64 12/17 <br />
Verifying : 1:openstack-packstack-puppet-7.0.0-0.7.dev1661.gaf13b7e.el7.noarch 13/17 <br />
Verifying : ruby-irb-2.0.0.598-25.el7_1.noarch 14/17 <br />
Verifying : rubygem-psych-2.0.0-25.el7_1.x86_64 15/17 <br />
Verifying : python-docutils-0.11-0.2.20130715svn7687.el7.noarch 16/17 <br />
Verifying : rubygem-io-console-0.4.2-25.el7_1.x86_64 17/17 <br />
*******************************************************************<br />
Dependency resolving was successful thanks to enabling these repositories:<br />
rhel-7-server-eus-optional-rpms<br />
*******************************************************************<br />
<br />
<br />
Installed:<br />
openstack-packstack.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 <br />
<br />
Dependency Installed:<br />
PyYAML.x86_64 0:3.10-11.el7 libyaml.x86_64 0:0.1.4-11.el7_0 <br />
openstack-packstack-puppet.noarch 1:7.0.0-0.7.dev1661.gaf13b7e.el7 openstack-puppet-modules.noarch 1:7.0.0-1.el7 <br />
python-docutils.noarch 0:0.11-0.2.20130715svn7687.el7 python-netaddr.noarch 0:0.7.18-1.el7 <br />
python-pillow.x86_64 0:2.0.0-19.gitd1c6db8.el7 ruby.x86_64 0:2.0.0.598-25.el7_1 <br />
ruby-irb.noarch 0:2.0.0.598-25.el7_1 ruby-libs.x86_64 0:2.0.0.598-25.el7_1 <br />
rubygem-bigdecimal.x86_64 0:1.2.0-25.el7_1 rubygem-io-console.x86_64 0:0.4.2-25.el7_1 <br />
rubygem-json.x86_64 0:1.7.7-25.el7_1 rubygem-psych.x86_64 0:2.0.0-25.el7_1 <br />
rubygem-rdoc.noarch 0:4.0.0-25.el7_1 rubygems.noarch 0:2.0.14-25.el7_1 <br />
<br />
Complete!<br />
[openstack@nuc0 ~]$ sudo packstack --allinone<br />
Welcome to the Packstack setup utility<br />
<br />
The installation log file is available at: /var/tmp/packstack/20151230-132844-2sQwfZ/openstack-setup.log<br />
Packstack changed given value to required value /root/.ssh/id_rsa.pub<br />
<br />
Installing:<br />
Clean Up [ DONE ]<br />
Discovering ip protocol version [ DONE ]<br />
Setting up ssh keys [ DONE ]<br />
Preparing servers [ DONE ]<br />
Pre installing Puppet and discovering hosts' details [ DONE ]<br />
Adding pre install manifest entries [ DONE ]<br />
Setting up CACERT [ DONE ]<br />
Adding AMQP manifest entries [ DONE ]<br />
Adding MariaDB manifest entries [ DONE ]<br />
Fixing Keystone LDAP config parameters to be undef if empty[ DONE ]<br />
Adding Keystone manifest entries [ DONE ]<br />
Adding Glance Keystone manifest entries [ DONE ]<br />
Adding Glance manifest entries [ DONE ]<br />
Adding Cinder Keystone manifest entries [ DONE ]<br />
Checking if the Cinder server has a cinder-volumes vg[ DONE ]<br />
Adding Cinder manifest entries [ DONE ]<br />
Adding Nova API manifest entries [ DONE ]<br />
Adding Nova Keystone manifest entries [ DONE ]<br />
Adding Nova Cert manifest entries [ DONE ]<br />
Adding Nova Conductor manifest entries [ DONE ]<br />
Creating ssh keys for Nova migration [ DONE ]<br />
Gathering ssh host keys for Nova migration [ DONE ]<br />
Adding Nova Compute manifest entries [ DONE ]<br />
Adding Nova Scheduler manifest entries [ DONE ]<br />
Adding Nova VNC Proxy manifest entries [ DONE ]<br />
Adding OpenStack Network-related Nova manifest entries[ DONE ]<br />
Adding Nova Common manifest entries [ DONE ]<br />
Adding Neutron VPNaaS Agent manifest entries [ DONE ]<br />
Adding Neutron FWaaS Agent manifest entries [ DONE ]<br />
Adding Neutron LBaaS Agent manifest entries [ DONE ]<br />
Adding Neutron API manifest entries [ DONE ]<br />
Adding Neutron Keystone manifest entries [ DONE ]<br />
Adding Neutron L3 manifest entries [ DONE ]<br />
Adding Neutron L2 Agent manifest entries [ DONE ]<br />
Adding Neutron DHCP Agent manifest entries [ DONE ]<br />
Adding Neutron Metering Agent manifest entries [ DONE ]<br />
Adding Neutron Metadata Agent manifest entries [ DONE ]<br />
Checking if NetworkManager is enabled and running [ DONE ]<br />
Adding OpenStack Client manifest entries [ DONE ]<br />
Adding Horizon manifest entries [ DONE ]<br />
Adding Swift Keystone manifest entries [ DONE ]<br />
Adding Swift builder manifest entries [ DONE ]<br />
Adding Swift proxy manifest entries [ DONE ]<br />
Adding Swift storage manifest entries [ DONE ]<br />
Adding Swift common manifest entries [ DONE ]<br />
Adding Provisioning Demo manifest entries [ DONE ]<br />
Adding Provisioning Demo bridge manifest entries [ DONE ]<br />
Adding Provisioning Glance manifest entries [ DONE ]<br />
Adding MongoDB manifest entries [ DONE ]<br />
Adding Redis manifest entries [ DONE ]<br />
Adding Ceilometer manifest entries [ DONE ]<br />
Adding Ceilometer Keystone manifest entries [ DONE ]<br />
Adding Nagios server manifest entries [ DONE ]<br />
Adding Nagios host manifest entries [ DONE ]<br />
Adding post install manifest entries [ DONE ]<br />
Copying Puppet modules and manifests [ DONE ]<br />
Applying 192.168.13.102_prescript.pp<br />
192.168.13.102_prescript.pp: [ DONE ] <br />
Applying 192.168.13.102_amqp.pp<br />
Applying 192.168.13.102_mariadb.pp<br />
192.168.13.102_amqp.pp: [ DONE ] <br />
192.168.13.102_mariadb.pp: [ DONE ] <br />
Applying 192.168.13.102_keystone.pp<br />
Applying 192.168.13.102_glance.pp<br />
Applying 192.168.13.102_cinder.pp<br />
192.168.13.102_keystone.pp: [ DONE ] <br />
192.168.13.102_glance.pp: [ DONE ] <br />
192.168.13.102_cinder.pp: [ DONE ] <br />
Applying 192.168.13.102_api_nova.pp<br />
192.168.13.102_api_nova.pp: [ DONE ] <br />
Applying 192.168.13.102_nova.pp<br />
192.168.13.102_nova.pp: [ DONE ] <br />
Applying 192.168.13.102_neutron.pp<br />
192.168.13.102_neutron.pp: [ DONE ] <br />
Applying 192.168.13.102_osclient.pp<br />
Applying 192.168.13.102_horizon.pp<br />
192.168.13.102_osclient.pp: [ DONE ] <br />
192.168.13.102_horizon.pp: [ DONE ] <br />
Applying 192.168.13.102_ring_swift.pp<br />
192.168.13.102_ring_swift.pp: [ DONE ] <br />
Applying 192.168.13.102_swift.pp<br />
Applying 192.168.13.102_provision_demo.pp<br />
192.168.13.102_swift.pp: [ DONE ] <br />
192.168.13.102_provision_demo.pp: [ DONE ] <br />
Applying 192.168.13.102_provision_demo_bridge.pp<br />
192.168.13.102_provision_demo_bridge.pp: [ DONE ] <br />
Applying 192.168.13.102_provision_glance<br />
192.168.13.102_provision_glance: [ DONE ] <br />
Applying 192.168.13.102_mongodb.pp<br />
Applying 192.168.13.102_redis.pp<br />
192.168.13.102_mongodb.pp: [ DONE ] <br />
192.168.13.102_redis.pp: [ DONE ] <br />
Applying 192.168.13.102_ceilometer.pp<br />
192.168.13.102_ceilometer.pp: [ DONE ] <br />
Applying 192.168.13.102_nagios.pp<br />
Applying 192.168.13.102_nagios_nrpe.pp<br />
192.168.13.102_nagios.pp: [ DONE ] <br />
192.168.13.102_nagios_nrpe.pp: [ DONE ] <br />
Applying 192.168.13.102_postscript.pp<br />
192.168.13.102_postscript.pp: [ DONE ] <br />
Applying Puppet manifests [ DONE ]<br />
Finalizing [ DONE ]<br />
<br />
**** Installation completed successfully ******<br />
<br />
Additional information:<br />
* A new answerfile was created in: /root/packstack-answers-20151230-132845.txt<br />
* Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.<br />
* Warning: NetworkManager is active on 192.168.13.102. OpenStack networking currently does not work on systems that have the Network Manager service enabled.<br />
* File /root/keystonerc_admin has been created on OpenStack client host 192.168.13.102. To use the command line tools you need to source the file.<br />
* To access the OpenStack Dashboard browse to http://192.168.13.102/dashboard .<br />
Please, find your login credentials stored in the keystonerc_admin in your home directory.<br />
* To use Nagios, browse to http://192.168.13.102/nagios username: nagiosadmin, password: 5462aceefa884a79<br />
* Because of the kernel update the host 192.168.13.102 requires reboot.<br />
* The installation log file is available at: /var/tmp/packstack/20151230-132844-2sQwfZ/openstack-setup.log<br />
* The generated manifests are available at: /var/tmp/packstack/20151230-132844-2sQwfZ/manifests<br />
[openstack@nuc0 ~]$ sudo cat /root/keystonerc_admin<br />
[sudo] password for openstack: <br />
unset OS_SERVICE_TOKEN<br />
export OS_USERNAME=admin<br />
export OS_PASSWORD=1a89f94c09fc419a<br />
export OS_AUTH_URL=http://192.168.13.102:5000/v2.0<br />
export PS1='[\u@\h \W(keystone_admin)]\$ '<br />
<br />
export OS_TENANT_NAME=admin<br />
export OS_REGION_NAME=RegionOne<br />
<br />
<br />
<h4>
install heat</h4>
<a href="https://www.rdoproject.org/install/deploy-heat-and-launch-your-first-application/">https://www.rdoproject.org/install/deploy-heat-and-launch-your-first-application/</a><br />
<br />
[root@nuc0 ~(keystone_admin)]# heat stack-list<br />
publicURL endpoint for orchestration service in RegionOne region not found<br />
[root@nuc0 ~(keystone_admin)]# exit<br />
logout<br />
[root@nuc0 openstack]# exit<br />
exit<br />
[openstack@nuc0 ~]$ sudo yum install "openstack-heat-*" python-heatclient<br />
[sudo] password for openstack:<br />
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager<br />
Package python-heatclient-0.8.0-1.el7.noarch already installed and latest version<br />
Resolving Dependencies<br />
--> Running transaction check<br />
---> Package openstack-heat-api.noarch 1:5.0.0-1.el7 will be installed<br />
---> Package openstack-heat-api-cfn.noarch 1:5.0.0-1.el7 will be installed<br />
---> Package openstack-heat-api-cloudwatch.noarch 1:5.0.0-1.el7 will be installed<br />
---> Package openstack-heat-common.noarch 1:5.0.0-1.el7 will be installed<br />
--> Processing Dependency: python-oslo-cache for package: 1:openstack-heat-common-5.0.0-1.el7.noarch<br />
---> Package openstack-heat-engine.noarch 1:5.0.0-1.el7 will be installed<br />
---> Package openstack-heat-templates.noarch 0:0-0.1.20151019.el7 will be installed<br />
--> Running transaction check<br />
---> Package python-oslo-cache.noarch 0:0.7.0-1.el7 will be installed<br />
--> Finished Dependency Resolution<br />
<br />
Dependencies Resolved<br />
<br />
==============================================================================================================================<br />
Package Arch Version Repository Size<br />
==============================================================================================================================<br />
Installing:<br />
openstack-heat-api noarch 1:5.0.0-1.el7 openstack-liberty 13 k<br />
openstack-heat-api-cfn noarch 1:5.0.0-1.el7 openstack-liberty 13 k<br />
openstack-heat-api-cloudwatch noarch 1:5.0.0-1.el7 openstack-liberty 13 k<br />
openstack-heat-common noarch 1:5.0.0-1.el7 openstack-liberty 1.1 M<br />
openstack-heat-engine noarch 1:5.0.0-1.el7 openstack-liberty 13 k<br />
openstack-heat-templates noarch 0-0.1.20151019.el7 openstack-liberty 71 k<br />
Installing for dependencies:<br />
python-oslo-cache noarch 0.7.0-1.el7 openstack-liberty 63 k<br />
<br />
Transaction Summary<br />
==============================================================================================================================<br />
Install 6 Packages (+1 Dependent package)<br />
<br />
Total download size: 1.3 M<br />
Installed size: 5.4 M<br />
Is this ok [y/d/N]: y<br />
Downloading packages:<br />
(1/7): openstack-heat-api-cfn-5.0.0-1.el7.noarch.rpm | 13 kB 00:00:00<br />
(2/7): openstack-heat-api-cloudwatch-5.0.0-1.el7.noarch.rpm | 13 kB 00:00:00<br />
(3/7): openstack-heat-api-5.0.0-1.el7.noarch.rpm | 13 kB 00:00:00<br />
(4/7): openstack-heat-engine-5.0.0-1.el7.noarch.rpm | 13 kB 00:00:00<br />
(5/7): openstack-heat-templates-0-0.1.20151019.el7.noarch.rpm | 71 kB 00:00:00<br />
(6/7): openstack-heat-common-5.0.0-1.el7.noarch.rpm | 1.1 MB 00:00:00<br />
(7/7): python-oslo-cache-0.7.0-1.el7.noarch.rpm | 63 kB 00:00:00<br />
------------------------------------------------------------------------------------------------------------------------------<br />
Total 954 kB/s | 1.3 MB 00:00:01<br />
Running transaction check<br />
Running transaction test<br />
Transaction test succeeded<br />
Running transaction<br />
Installing : python-oslo-cache-0.7.0-1.el7.noarch 1/7<br />
Installing : 1:openstack-heat-common-5.0.0-1.el7.noarch 2/7<br />
Installing : 1:openstack-heat-api-cfn-5.0.0-1.el7.noarch 3/7<br />
Installing : 1:openstack-heat-api-cloudwatch-5.0.0-1.el7.noarch 4/7<br />
Installing : 1:openstack-heat-api-5.0.0-1.el7.noarch 5/7<br />
Installing : 1:openstack-heat-engine-5.0.0-1.el7.noarch 6/7<br />
Installing : openstack-heat-templates-0-0.1.20151019.el7.noarch 7/7<br />
Verifying : 1:openstack-heat-api-cfn-5.0.0-1.el7.noarch 1/7<br />
Verifying : 1:openstack-heat-api-cloudwatch-5.0.0-1.el7.noarch 2/7<br />
Verifying : 1:openstack-heat-api-5.0.0-1.el7.noarch 3/7<br />
Verifying : 1:openstack-heat-common-5.0.0-1.el7.noarch 4/7<br />
Verifying : openstack-heat-templates-0-0.1.20151019.el7.noarch 5/7<br />
Verifying : 1:openstack-heat-engine-5.0.0-1.el7.noarch 6/7<br />
Verifying : python-oslo-cache-0.7.0-1.el7.noarch 7/7<br />
<br />
Installed:<br />
openstack-heat-api.noarch 1:5.0.0-1.el7 openstack-heat-api-cfn.noarch 1:5.0.0-1.el7<br />
openstack-heat-api-cloudwatch.noarch 1:5.0.0-1.el7 openstack-heat-common.noarch 1:5.0.0-1.el7<br />
openstack-heat-engine.noarch 1:5.0.0-1.el7 openstack-heat-templates.noarch 0:0-0.1.20151019.el7<br />
<br />
Dependency Installed:<br />
python-oslo-cache.noarch 0:0.7.0-1.el7<br />
<br />
Complete!<br />
<br />
[root@biometric91 ~]# openstack-status<br />
== Nova services ==<br />
openstack-nova-api: active<br />
openstack-nova-cert: active<br />
openstack-nova-compute: active<br />
openstack-nova-network: inactive (disabled on boot)<br />
openstack-nova-scheduler: active<br />
openstack-nova-conductor: active<br />
== Glance services ==<br />
openstack-glance-api: active<br />
openstack-glance-registry: active<br />
== Keystone service ==<br />
openstack-keystone: inactive (disabled on boot)<br />
== Horizon service ==<br />
openstack-dashboard: active<br />
== neutron services ==<br />
neutron-server: active<br />
neutron-dhcp-agent: active<br />
neutron-l3-agent: active<br />
neutron-metadata-agent: active<br />
neutron-openvswitch-agent: active<br />
== Swift services ==<br />
openstack-swift-proxy: active<br />
openstack-swift-account: active<br />
openstack-swift-container: active<br />
openstack-swift-object: active<br />
== Cinder services ==<br />
openstack-cinder-api: active<br />
openstack-cinder-scheduler: active<br />
openstack-cinder-volume: active<br />
openstack-cinder-backup: active<br />
== Ceilometer services ==<br />
openstack-ceilometer-api: active<br />
openstack-ceilometer-central: active<br />
openstack-ceilometer-compute: active<br />
openstack-ceilometer-collector: active<br />
openstack-ceilometer-alarm-notifier: active<br />
openstack-ceilometer-alarm-evaluator: active<br />
openstack-ceilometer-notification: active<br />
== Support services ==<br />
mysqld: inactive (disabled on boot)<br />
openvswitch: active<br />
dbus: active<br />
target: active<br />
rabbitmq-server: active<br />
memcached: active<br />
<div>
<br /></div>
<br />
<h3>
Ubuntu Canonical MAAS</h3>
http://ronaldbradford.com/blog/installing-ubuntu-openstack-2015-06-01/<br />
<pre style="background-color: #eeeeee; border-radius: 4px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 11px; line-height: 15px; margin-bottom: 10px; margin-top: 10px; max-height: 300px; overflow: auto; padding: 9.5px 15px; width: 741px; word-break: break-all; word-wrap: break-word;">sudo apt-add-repository -y ppa:cloud-installer/stable
sudo apt-get update
sudo apt-get install -y openstack
sudo openstack-install --version
sudo openstack-install</pre>
works but no web access without<br />
openstack@biomos14:~$ sudo iptables -t nat -A PREROUTING -p tcp -i em1 --dport 80 -j DNAT --to-destination 10.0.6.4:80<br />
openstack@biomos14:~$ sudo iptables -A FORWARD -p tcp -d 10.0.6.4 --dport 80 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT<br />
<br />
add heat via<br />
<a href="http://openstack.astokes.org/">http://openstack.astokes.org/</a><br />
<a href="https://wiki.ubuntu.com/OpenStack/Installer/using-juju">https://wiki.ubuntu.com/OpenStack/Installer/using-juju</a><br />
<br />
38 openstack-juju status<br />
39 openstack-juju deploy heat<br />
<div>
<br /></div>
<div>
<br />
restarting<br />
<a href="https://help.ubuntu.com/lts/clouddocs/installer/en/single-install.html">https://help.ubuntu.com/lts/clouddocs/installer/en/single-install.html</a><br />
does not work because of missing bridge<br />
<br />
openstack@biomos14:~$ sudo lxc-start -n openstack-single-openstack<br />
lxc-start: conf.c: instantiate_veth: 3105 failed to attach 'vethVED3WO' to the bridge 'lxcbr0': No such device<br />
<div>
<br /></div>
<div>
Above runs lxc vms</div>
<div>
<a href="http://ilearnstack.com/2013/04/26/setting-up-a-single-node-openstack-environment/">http://ilearnstack.com/2013/04/26/setting-up-a-single-node-openstack-environment/</a></div>
<div>
<br /></div>
fails on<br />
Adding new user `rabbitmq' (UID 111) with group `rabbitmq' ...<br />
Not creating home directory `/var/lib/rabbitmq'.<br />
* Starting message broker rabbitmq-server<br />
* FAILED - check /var/log/rabbitmq/startup_\{log, _err\}<br />
...fail!<br />
invoke-rc.d: initscript rabbitmq-server, action "start" failed.<br />
dpkg: error processing package rabbitmq-server (--configure):<br />
subprocess installed post-installation script returned error exit status 1<br />
Processing triggers for libc-bin (2.19-0ubuntu6.6) ...<br />
Processing triggers for ureadahead (0.100.0-16) ...<br />
Errors were encountered while processing:<br />
rabbitmq-server<br />
E: Sub-process /usr/bin/dpkg returned an error code (1)<br />
Error None<br />
Failed to execute command apt-get install rabbitmq-server -y<br />
100<br />
Traceback (most recent call last):<br />
File "cloudgear.py", line 404, in <module><br />
install_rabbitmq()<br />
File "cloudgear.py", line 135, in install_rabbitmq<br />
execute("apt-get install rabbitmq-server -y", True)<br />
File "cloudgear.py", line 100, in execute<br />
raise Exception(output)<br />
Exception<br />
<div>
<br /></div>
<div>
on </div>
<div>
apt-get install rabbitmq-server -y</div>
<div>
removing</div>
<div>
<div>
root@biomos14:~/cloudgear# apt-get remove rabbitmq-server</div>
<div>
Reading package lists... Done</div>
<div>
Building dependency tree</div>
<div>
Reading state information... Done</div>
<div>
The following packages were automatically installed and are no longer required:</div>
<div>
erlang-asn1 erlang-base erlang-corba erlang-crypto erlang-diameter</div>
<div>
erlang-edoc erlang-eldap erlang-erl-docgen erlang-eunit erlang-ic</div>
<div>
erlang-inets erlang-mnesia erlang-nox erlang-odbc erlang-os-mon</div>
<div>
erlang-parsetools erlang-percept erlang-public-key erlang-runtime-tools</div>
<div>
erlang-snmp erlang-ssh erlang-ssl erlang-syntax-tools erlang-tools</div>
<div>
erlang-webtool erlang-xmerl libodbc1 libsctp1 lksctp-tools</div>
<div>
Use 'apt-get autoremove' to remove them.</div>
<div>
The following packages will be REMOVED:</div>
<div>
rabbitmq-server</div>
<div>
0 upgraded, 0 newly installed, 1 to remove and 88 not upgraded.</div>
<div>
1 not fully installed or removed.</div>
<div>
After this operation, 4,867 kB disk space will be freed.</div>
<div>
Do you want to continue? [Y/n] y</div>
<div>
(Reading database ... 80922 files and directories currently installed.)</div>
<div>
Removing rabbitmq-server (3.2.4-1) ...</div>
<div>
* Stopping message broker rabbitmq-server * message broker already stopped</div>
<div>
</div>
</div>
<div>
<div>
root@biomos14:~/cloudgear# service rabbitmq-server restart</div>
<div>
* Restarting message broker rabbitmq-server [fail]</div>
</div>
<div>
<br /></div>
<div>
<a href="http://ronaldbradford.com/blog/tracking-the-ubuntu-openstack-installation-process-2015-06-02/">http://ronaldbradford.com/blog/tracking-the-ubuntu-openstack-installation-process-2015-06-02/</a></div>
<div>
<br /></div>
<table class="table table-bordered table-striped datatable tablesorter tablesorter-default" id="services" style="background-color: white; border-collapse: collapse; border-spacing: 0px; border: none; box-sizing: border-box; color: #333333; font-family: Ubuntu, Arial, 'libra sans', sans-serif; font-size: 13px; line-height: 18.5714px; margin-bottom: 25px; max-width: 100%; width: 1012px;"><thead style="box-sizing: border-box;">
<tr class="tablesorter-headerRow" style="box-sizing: border-box;"><th class="sortable normal_column tablesorter-header" data-column="1" style="background: right 5px top 50% no-repeat rgb(241, 241, 241); border-color: rgb(221, 221, 221); border-style: solid; border-top-left-radius: 4px; border-width: 1px 1px 2px; box-sizing: border-box; cursor: pointer; line-height: 1.42857; padding: 8px; vertical-align: middle;" tabindex="0"><div class="tablesorter-header-inner" style="box-sizing: border-box;">
Name</div>
</th><th class="sortable normal_column tablesorter-header" data-column="2" style="background: right 5px top 50% no-repeat rgb(241, 241, 241); border-color: rgb(221, 221, 221); border-style: solid; border-width: 1px 1px 2px; box-sizing: border-box; cursor: pointer; line-height: 1.42857; padding: 8px; vertical-align: middle;" tabindex="0"><div class="tablesorter-header-inner" style="box-sizing: border-box;">
Service</div>
</th><th class="sortable normal_column tablesorter-header" data-column="3" style="background: right 5px top 50% no-repeat rgb(241, 241, 241); border-color: rgb(221, 221, 221); border-style: solid; border-width: 1px 1px 2px; box-sizing: border-box; cursor: pointer; line-height: 1.42857; padding: 8px; vertical-align: middle;" tabindex="0"><div class="tablesorter-header-inner" style="box-sizing: border-box;">
Host</div>
</th><th class="sortable normal_column tablesorter-header" data-column="4" style="background: right 5px top 50% no-repeat rgb(241, 241, 241); border-color: rgb(221, 221, 221); border-style: solid; border-top-right-radius: 4px; border-width: 1px 1px 2px; box-sizing: border-box; cursor: pointer; line-height: 1.42857; padding: 8px; vertical-align: middle;" tabindex="0"><div class="tablesorter-header-inner" style="box-sizing: border-box;">
Status</div>
</th></tr>
</thead><tbody style="box-sizing: border-box;">
<tr class="status_up odd" data-display="image-stream" data-object-id="0" id="services__row__0" style="box-sizing: border-box;"><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">image-stream</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">product-streams</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">10.0.6.110</td><td class="status_up sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">Enabled</td></tr>
<tr class="status_up even" data-display="nova" data-object-id="1" id="services__row__1" style="box-sizing: border-box;"><td class="sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">nova</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">compute</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">10.0.6.245</td><td class="status_up sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">Enabled</td></tr>
<tr class="status_up odd" data-display="glance" data-object-id="2" id="services__row__2" style="box-sizing: border-box;"><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">glance</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">image</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">10.0.6.42</td><td class="status_up sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">Enabled</td></tr>
<tr class="status_up even" data-display="keystone" data-object-id="3" id="services__row__3" style="box-sizing: border-box;"><td class="sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">keystone</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">identity (native backend)</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">10.0.6.206</td><td class="status_up sortable normal_column" style="background-clip: padding-box; background-color: inherit; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">Enabled</td></tr>
<tr class="status_up odd" data-display="quantum" data-object-id="4" id="services__row__4" style="box-sizing: border-box;"><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">quantum</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">network</td><td class="sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">10.0.6.145</td><td class="status_up sortable normal_column" style="background-clip: padding-box; background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; line-height: 1.42857; padding: 8px; transition: background 0.2s; vertical-align: middle;">Enabled</td></tr>
</tbody><tfoot style="box-sizing: border-box;">
<tr style="box-sizing: border-box;"><td colspan="5" data-column="0" style="background-color: #f1f1f1; border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; border: 1px solid rgb(221, 221, 221); box-sizing: border-box; font-size: 11px; line-height: 14px; padding: 8px; transition: background 0.2s; vertical-align: middle;"><span class="table_count" style="box-sizing: border-box;">Displaying 5 items</span></td></tr>
</tfoot></table>
</div>
<h2 style="border-bottom-color: rgb(204, 204, 204); color: #333333; font-family: Arial, sans-serif; font-size: 20px; font-weight: normal; line-height: 1.5; margin: 30px 0px 0px;">
Devstack Procedure</h2>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
Read <a data-mce-href="http://docs.openstack.org/developer/devstack/" href="http://docs.openstack.org/developer/devstack/" style="color: rgb(59, 115, 175) !important; text-decoration: none;">http://docs.openstack.org/developer/devstack/</a></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
Get some hardware - 1 box required here (i7-5820K 4500MHz, 64G RAM, 256g SSD (raid0 drives not recognized), 2 NICs)</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
Install Ubuntu 14.04 - <a data-mce-href="http://www.ubuntu.com/download/server" href="http://www.ubuntu.com/download/server" style="color: rgb(59, 115, 175) !important; text-decoration: none;">http://www.ubuntu.com/download/server</a></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
login as: openstack<br />
<a data-mce-href="mailto:openstack@138.120.143.140" href="mailto:openstack@138.120.143.140" style="color: rgb(59, 115, 175) !important; text-decoration: none;">openstack@138.120.143.140</a>'s password:<br />
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.19.0-25-generic x86_64)</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
* Documentation: <a data-mce-href="https://help.ubuntu.com/" href="https://help.ubuntu.com/" style="color: rgb(59, 115, 175) !important; text-decoration: none;">https://help.ubuntu.com/</a></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
System information as of Tue Dec 22 11:11:28 EST 2015</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
System load: 1.5 Processes: 222<br />
Usage of /: 0.4% of 202.96GB Users logged in: 0<br />
Memory usage: 0% IP address for em1: 138.120.143.140<br />
Swap usage: 0% IP address for eth0: 138.120.140.185</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
Login: openstack:mainstreet</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~$ ifconfig<br />
em1 Link encap:Ethernet HWaddr f0:79:59:e0:09:c4<br />
inet addr:138.120.143.140 Bcast:138.120.143.255 Mask:255.255.252.0</div>
<h3 style="color: #333333; font-family: Arial, sans-serif; font-size: 16px; line-height: 1.5; margin: 30px 0px 0px;">
Configure Networks</h3>
<h4 style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin: 10px 0px 0px;">
get interfaces</h4>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~$ ip link show</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default<br />
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br />
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000<br />
link/ether f0:79:59:e0:09:c4 brd ff:ff:ff:ff:ff:ff<br />
3: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000<br />
link/ether ec:22:80:a3:49:88 brd ff:ff:ff:ff:ff:ff</div>
<h4 style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin: 20px 0px 0px;">
enable 2nd interface (eth0)</h4>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
>sudo vi /etc/network/interfaces</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
+auto eth0<br />
+iface eth0 inet dhcp</div>
<h4 style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin: 20px 0px 0px;">
restart networking services</h4>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
>openstack@biomos00:~$ sudo reboot now</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~$ ifconfig<br />
em1 Link encap:Ethernet HWaddr f0:79:59:e0:09:c4<br />
inet addr:138.120.143.140 Bcast:138.120.143.255 Mask:255.255.252.0<br />
eth0 Link encap:Ethernet HWaddr ec:22:80:a3:49:88<br />
inet addr:138.120.140.185 Bcast:138.120.143.255 Mask:255.255.252.0<br />
lo Link encap:Local Loopback<br />
inet addr:127.0.0.1 Mask:255.0.0.0</div>
<h3 style="color: #333333; font-family: Arial, sans-serif; font-size: 16px; line-height: 1.5; margin: 30px 0px 0px;">
Verify proxy</h3>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~$ vi ~/.bashrc</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
export no_proxy=localhost,127.0.0.1<br />
export http_proxy=<a data-mce-href="http://global.proxy.alcatel-lucent.com:8000" href="http://global.proxy/" style="color: rgb(59, 115, 175) !important; text-decoration: none;">http://global.proxy</a><br />
export https_proxy=<a data-mce-href="http://global.proxy.alcatel-lucent.com:8000" href="http://global.proxy/" style="color: rgb(59, 115, 175) !important; text-decoration: none;">http://global.proxy</a></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
<br />
openstack@biomos00:~$ sudo vi /etc/apt/apt.conf<br />
[sudo] password for openstack:<br />
Acquire::http::Proxy "<a data-mce-href="http://global.proxy.alcatel-lucent.com:8000" href="http://global.proxy/" style="color: rgb(59, 115, 175) !important; text-decoration: none;">http://global.proxy</a>";</div>
<h3 style="color: #333333; font-family: Arial, sans-serif; font-size: 16px; line-height: 1.5; margin: 30px 0px 0px;">
update and install git</h3>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~$ sudo apt-get update</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
Fetched 35.7 MB in 26s (1,345 kB/s)<br />
Reading package lists... Done</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~$ sudo apt-get install git</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
<br /></div>
<h3 style="color: #333333; font-family: Arial, sans-serif; font-size: 16px; line-height: 1.5; margin: 30px 0px 0px;">
get the devstack repo</h3>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~$ git clone <a data-mce-href="https://github.com/openstack-dev/devstack" data-mce-style="line-height: 1.42857;" href="https://github.com/openstack-dev/devstack" style="color: rgb(59, 115, 175) !important; line-height: 1.42857; text-decoration: none;">https://github.com/openstack-dev/devstack</a></div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
Cloning into 'devstack'...<br />
remote: Counting objects: 31195, done.</div>
<h3 style="color: #333333; font-family: Arial, sans-serif; font-size: 16px; line-height: 1.5; margin: 30px 0px 0px;">
Change ownership and permissions</h3>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
<br />
openstack@biomos00:~$ sudo chown -R openstack devstack<br />
openstack@biomos00:~$ sudo chmod 770 devstack<br />
openstack@biomos00:~$ cd devstack</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
You will get the following inside our firewall because we glock port 9418 for git://</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
fatal: unable to access '<a data-mce-href="https://github.com/openstack-dev/devstack/" href="https://github.com/openstack-dev/devstack/" style="color: rgb(59, 115, 175) !important; text-decoration: none;">https://github.com/openstack-dev/devstack/</a>': Received HTTP code 503 from proxy after CONNECT</div>
<h3 style="color: #333333; font-family: Arial, sans-serif; font-size: 16px; line-height: 1.5; margin: 30px 0px 0px;">
modify git to use https:// to avoid the firewall</h3>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
openstack@biomos00:~/devstack$ sudo vi stackrc<br />
#GIT_BASE=${GIT_BASE:<a data-mce-style="line-height: 1.42857;" href="https://www.blogger.com/null" style="color: rgb(59, 115, 175) !important; line-height: 1.42857;">-git://git.openstack.org</a>}<br />
GIT_BASE=${GIT_BASE:<a data-mce-style="line-height: 1.42857;" href="https://www.blogger.com/null" style="color: rgb(59, 115, 175) !important; line-height: 1.42857;">-https://git.openstack.org</a>}</div>
<h3 style="color: #333333; font-family: Arial, sans-serif; font-size: 16px; line-height: 1.5; margin: 30px 0px 0px;">
Install devstack </h3>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
use "admin" for all the passwords</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
<br />
openstack@biomos00:~/devstack$ ./stack.sh</div>
<div style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 18.1818px; margin-top: 10px;">
Duration 45 min<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuOY_kI6USWO7KrAZqqLJhFb8qOU__tAvm3Z79cA-ymZ4YOWaWZ5SX_GlEn2NP1fmTMmKVwDa5xUxFn85GP8IWkmGj1Leic8LJXAsyE5LF7xgu84nTHyXOVa9yU1l6Tin6MeqboZk5Yge5/s1600/20151222_openstack_liberty_hypervisors.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="372" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuOY_kI6USWO7KrAZqqLJhFb8qOU__tAvm3Z79cA-ymZ4YOWaWZ5SX_GlEn2NP1fmTMmKVwDa5xUxFn85GP8IWkmGj1Leic8LJXAsyE5LF7xgu84nTHyXOVa9yU1l6Tin6MeqboZk5Yge5/s640/20151222_openstack_liberty_hypervisors.png" width="640" /></a></div>
<br /></div>
<pre class="prettyprint" style="background-color: whitesmoke; border-radius: 4px; border: 1px solid rgb(136, 136, 136); clear: both; color: #333333; font-family: Menlo, Monaco, 'Courier New', monospace; font-size: 12.025px; line-height: 18px; margin-bottom: 18px; padding: 2px; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><code style="background-color: transparent; border-radius: 3px; border: 0px; clear: both; color: inherit; font-family: Menlo, Monaco, 'Courier New', monospace; font-size: 12px; padding: 0px;"><span class="pln" style="color: black;">
</span></code></pre>
<pre class="prettyprint" style="background-color: whitesmoke; border-radius: 4px; border: 1px solid rgb(136, 136, 136); clear: both; color: #333333; font-family: Menlo, Monaco, 'Courier New', monospace; font-size: 12.025px; line-height: 18px; margin-bottom: 18px; padding: 2px; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><code style="background-color: transparent; border-radius: 3px; border: 0px; clear: both; padding: 0px;"><span class="pln" style="font-family: "menlo" , "monaco" , "courier new" , monospace; font-size: 12px;">openstack@biomos300ds:~$ history
1 sudo apt-get update
2 sudo apt-get install git
3 git clone https://git.openstack.org/openstack-dev/devstack
4 ls
5 chmod -R 770 devstack
6 sudo chown -R openstack devstack
12 cp samples/local.conf .
13 sudo vi local.conf
SERVICE_TOKEN=azertytoken
ADMIN_PASSWORD=nomoresecrete
DATABASE_PASSWORD=stackdb
RABBIT_PASSWORD=stackqueue
SERVICE_PASSWORD=$ADMIN_PASSWORD</span></code></pre>
<pre class="prettyprint" style="background-color: whitesmoke; border-radius: 4px; border: 1px solid rgb(136, 136, 136); clear: both; color: #333333; font-family: Menlo, Monaco, 'Courier New', monospace; font-size: 12.025px; line-height: 18px; margin-bottom: 18px; padding: 2px; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-86256721002929825172015-06-04T13:15:00.001-07:002017-05-13T20:54:50.640-07:00Microservices using AWS Beanstalk or ECS for Docker<div>
<h2>
Docker</h2>
<a href="http://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html">http://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html</a></div>
<div>
<a href="https://wiki.openstack.org/wiki/Docker">https://wiki.openstack.org/wiki/Docker</a></div>
<div>
<br /></div>
<div>
<a href="https://docs.docker.com/userguide/">https://docs.docker.com/userguide/</a></div>
<div>
<a href="http://www.kegel.com/c10k.html">http://www.kegel.com/c10k.html</a></div>
<div>
<a href="http://wiki.nginx.org/Main">http://wiki.nginx.org/Main</a><br />
Services discovery<br />
<a href="https://www.consul.io/">https://www.consul.io/</a></div>
<div>
<a href="https://registry.hub.docker.com/u/progrium/consul/">https://registry.hub.docker.com/u/progrium/consul/</a><br />
<br />
<h3>
Docker Compose</h3>
<h3>
Docker commands</h3>
<div>
getting a shell</div>
<div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">$ docker exec -lt tomcat bash</span></div>
<div class="p1">
<span class="s1">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #ffffff; background-color: #2b66c9}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
</span></div>
<div class="p1">
<span class="s1">$ winpty docker exec -lt tomcat bash</span></div>
</div>
<h3>
Docker Installation on RHEL 7.2</h3>
<div>
201705 update</div>
<div>
<div>
77 sudo yum install docker</div>
<div>
78 sudo yum install -y yum-utils</div>
<div>
79 sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo</div>
<div>
80 sudo yum-config-manager --enable docker-ce-edge</div>
<div>
81 sudo yum makecache fast</div>
<div>
83 sudo yum install docker-ce</div>
<div>
84 sudo systemctl start docker</div>
<div>
85 sudo docker run hello-world</div>
<div>
86 docker ps</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div class="p1">
<span class="s1"><a href="https://blog.docker.com/2015/07/new-apt-and-yum-repos/">https://blog.docker.com/2015/07/new-apt-and-yum-repos/</a></span><br />
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
We are moving to microservices. This architectural change will involve partitioning our code along service lines that will also break up our current WAR based deployments. Part of this exercise accomplished via a move to Docker containers and Kubernetes as the orchestration and auto scaling engine. The docker containers will live in our current OpenStack private cloud.</div>
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span style="font-size: 20px; line-height: 1.5;">Installing Docker</span></div>
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
Install docker on RHEL 7.2 (7.x supported)<br />
<a class="external-link" href="https://docs.docker.com/engine/installation/rhel/" rel="nofollow" style="color: #3b73af; text-decoration: none;">https://docs.docker.com/engine/installation/rhel/</a><br />
Set Alcatel-Lucent proxy<br />
<a class="external-link" href="https://docs.docker.com/engine/articles/systemd/#http-proxy" rel="nofollow" style="color: #3b73af; text-decoration: none;">https://docs.docker.com/engine/articles/systemd/#http-proxy</a></div>
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span style="font-size: 16px; line-height: 1.5;">Installing on a RHEL 7.2 metal or VM instance</span></div>
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
sudo su</div>
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
mkdir /etc/systemd/system/docker.service.d</div>
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
vi /etc/systemd/system/docker.service.d/http-proxy.conf</div>
<div class="table-wrap" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin: 10px 0px 0px; overflow-x: auto; padding: 0px;">
<table class="confluenceTable" style="border-collapse: collapse; margin: 0px; overflow-x: auto;"><tbody>
<tr><th class="confluenceTh" style="background-color: #f0f0f0; border: 1px solid rgb(221, 221, 221); color: #333333; padding: 7px 10px; vertical-align: top;"><div class="p1" style="padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">[Service]</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">Environment="HTTP_PROXY=http://global.proxy:80"</span></span></div>
</th></tr>
</tbody></table>
</div>
<div style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
vi /etc/yum.repos.d/docker.repo</div>
<div class="table-wrap" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin: 10px 0px 0px; overflow-x: auto; padding: 0px;">
<table class="confluenceTable" style="border-collapse: collapse; margin: 0px; overflow-x: auto;"><tbody>
<tr><th class="confluenceTh" style="background-color: #f0f0f0; border: 1px solid rgb(221, 221, 221); color: #333333; padding: 7px 10px; vertical-align: top;"><div class="p1" style="padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: xx-small;">[dockerrepo]</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: xx-small;">name=Docker Repository</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: xx-small;">baseurl=<a class="external-link" href="https://yum.dockerproject.org/repo/main/centos/7" rel="nofollow" style="color: #3b73af; text-decoration: none;">https://yum.dockerproject.org/repo/main/centos/7</a></span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: xx-small;">enabled=1</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: xx-small;">gpgcheck=1</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: xx-small;">gpgkey=<a class="external-link" href="https://yum.dockerproject.org/gpg" rel="nofollow" style="color: #3b73af; text-decoration: none;">https://yum.dockerproject.org/gpg</a></span></span></div>
</th></tr>
</tbody></table>
</div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
yum install docker-engine</div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
service docker start</div>
<h2 class="p1" id="Docker-Verifycontaineroperation" style="background-color: white; border-bottom-color: rgb(204, 204, 204); color: #333333; font-family: Arial, sans-serif; font-size: 20px; font-weight: normal; line-height: 1.5; margin: 30px 0px 0px; padding: 0px;">
Verify container operation</h2>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1">sudo docker run hello-world</span></div>
<div class="table-wrap" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin: 10px 0px 0px; overflow-x: auto; padding: 0px;">
<table class="confluenceTable" style="border-collapse: collapse; margin: 0px; overflow-x: auto;"><tbody>
<tr><th class="confluenceTh" style="background-color: #f0f0f0; border: 1px solid rgb(221, 221, 221); color: #333333; padding: 7px 10px; vertical-align: top;"><br /></th></tr>
</tbody></table>
</div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1"><br /></span></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1">A more realistic demo would be to create an empty fedora (RHEL compatible) vm and enter a shell on the VM. Open a 2nd cmd window and type "docker events" to listen in on container creation/destruction - in the case where you "<b>exit</b>" from the container.</span></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<i>in a 1st terminal</i></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1">obrienlabs-mbp15:~ michaelobrien$ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no <a class="external-link" href="mailto:stack@192.168.121.133" rel="nofollow" style="color: #3b73af; text-decoration: none;">root@192.168.121.133</a></span></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1">docker events</span></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<i>and in a 2nd terminal</i></div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1">docker run --rm -ti fedora:latest /bin/bash</span></div>
<div class="table-wrap" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin: 10px 0px 0px; overflow-x: auto; padding: 0px;">
<table class="confluenceTable" style="border-collapse: collapse; margin: 0px; overflow-x: auto;"><tbody>
<tr><th class="confluenceTh" style="background-color: #f0f0f0; border: 1px solid rgb(221, 221, 221); color: #333333; padding: 7px 10px; vertical-align: top;"><div class="p1" style="padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">[root@rhel72virtual stack]# docker run --rm -ti fedora:latest /bin/bash </span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">Unable to find image 'fedora:latest' locally </span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">latest: Pulling from library/fedora</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">369aca82a5c0: Pull complete </span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">3fc68076e184: Pull complete </span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">Digest: sha256:7d916d5d3ab2d92147e515d3c09cb47a0431e2fff9d550fd9bcc4fed379af9ea</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">Status: Downloaded newer image for fedora:latest</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">[root@e7de408df84b /]# ls</span></span></div>
<div class="p3" style="margin-top: 10px; padding: 0px; text-align: left;">
<span style="font-size: x-small;"><span class="s2">bin</span><span class="s3"> </span><span class="s1">boot</span><span class="s3"> </span><span class="s1">dev</span><span class="s3"> </span><span class="s1">etc</span><span class="s3"> </span><span class="s1">home</span><span class="s3"> </span><span class="s2">lib</span><span class="s3"> </span><span class="s2">lib64</span><span class="s3"> </span><span class="s1">lost+found</span><span class="s3"> </span><span class="s1">media</span><span class="s3"> </span><span class="s1">mnt</span><span class="s3"> </span><span class="s1">opt</span><span class="s3"> </span><span class="s1">proc</span><span class="s3"> </span><span class="s1">root</span><span class="s3"> </span><span class="s1">run</span><span class="s3"> </span><span class="s2">sbin</span><span class="s3"> </span><span class="s1">srv</span><span class="s3"> </span><span class="s1">sys</span><span class="s3"> </span><span class="s4">tmp</span><span class="s3"> </span><span class="s1">usr</span><span class="s3"> </span><span class="s1">var</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">[root@e7de408df84b /]# df</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">Filesystem 1K-blocks Used Available Use% Mounted on</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">/dev/mapper/docker-253:0-21844-e7de408df84b187a90e298609097a27dd1eeda0bea346278d3ebff2343e98705 104805376 255332 104550044 1% /</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">[root@e7de408df84b /]# exit</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;"> exit</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">[root@rhel72virtual stack]# </span></span></div>
</th></tr>
</tbody></table>
</div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1">in the "docker events" terminal (note: the hex is different because I tested with a 2nd instance</span></div>
<div class="table-wrap" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin: 10px 0px 0px; overflow-x: auto; padding: 0px;">
<table class="confluenceTable" style="border-collapse: collapse; margin: 0px; overflow-x: auto;"><tbody>
<tr><th class="confluenceTh" style="background-color: #f0f0f0; border: 1px solid rgb(221, 221, 221); color: #333333; padding: 7px 10px; vertical-align: top;"><div class="p1" style="padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">[root@rhel72virtual ~]# docker events</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">2016-01-05T17:04:37.916688150-05:00 b633da3a3e55457cb7b81d75f0516f07da065d211b07573b4ae79d5762322f4a: (from fedora:latest) create</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">2016-01-05T17:04:37.951828323-05:00 b633da3a3e55457cb7b81d75f0516f07da065d211b07573b4ae79d5762322f4a: (from fedora:latest) attach</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">2016-01-05T17:04:38.006487503-05:00 b633da3a3e55457cb7b81d75f0516f07da065d211b07573b4ae79d5762322f4a: (from fedora:latest) start</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">2016-01-05T17:04:38.147536237-05:00 b633da3a3e55457cb7b81d75f0516f07da065d211b07573b4ae79d5762322f4a: (from fedora:latest) resize</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">2016-01-05T17:04:56.375942258-05:00 b633da3a3e55457cb7b81d75f0516f07da065d211b07573b4ae79d5762322f4a: (from fedora:latest) die</span></span></div>
<div class="p1" style="margin-top: 10px; padding: 0px; text-align: left;">
<span class="s1"><span style="font-size: x-small;">2016-01-05T17:04:58.110736214-05:00 b633da3a3e55457cb7b81d75f0516f07da065d211b07573b4ae79d5762322f4a: (from fedora:latest) destroy</span></span></div>
</th></tr>
</tbody></table>
</div>
<div class="p1" style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="s1"><br /></span></div>
</div>
<h3>
Docker Installation on Ubuntu 14.04</h3>
<br />
<br />
<h3>
Docker Installation on OSX 10.11</h3>
<br />
The following tutorial is very good<br />
<a href="https://docs.docker.com/installation/mac/">https://docs.docker.com/installation/mac/</a></div>
<div>
<div class="p1">
<span class="s1">bash-3.2$ docker run -d -P --name web nginx</span></div>
<div class="p1">
<span class="s1">bash-3.2$ docker ps</span></div>
<div class="p1">
<span class="s1">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span></div>
<div class="p1">
<span class="s1"></span></div>
<div class="p1">
<span class="s1">15abd6456e7e nginx:latest "nginx -g 'daemon of 56 seconds ago Up 56 seconds 0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp web </span></div>
<div class="p1">
<span class="s1"></span></div>
<div class="p1">
<span class="s1">bash-3.2$ docker port web</span></div>
<div class="p1">
<span class="s1"></span></div>
<div class="p1">
<span class="s1">bash-3.2$ boot2docker ip</span></div>
<div class="p1">
<span class="s1"><a href="http://192.168.59.103:32769/">http://192.168.59.103:32769</a></span><br />
<div class="p1">
<span class="s1">bash-3.2$ docker run -d -P -v $HOME/site:/usr/share/nginx/html --name mysite nginx</span></div>
<br />
<div class="p1">
<span class="s1">e6b5188e2dba301ce9b041b472dc9dc79272beae87d9840c82052ddd6735e614</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">bash-3.2$ docker ps</span></div>
<div class="p1">
<span class="s1">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span></div>
<div class="p1">
<span class="s1">34e4e4239aa7 nginx:latest "nginx -g 'daemon of 6 minutes ago Up 6 minutes 0.0.0.0:32777->80/tcp, 0.0.0.0:32776->443/tcp mysite </span></div>
<div class="p1">
<span class="s1">
</span></div>
<div class="p1">
<span class="s1">15abd6456e7e nginx:latest "nginx -g 'daemon of 32 minutes ago Up 32 minutes 0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp web </span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">bash-3.2$ docker stop mysite</span></div>
<br />
<div class="p1">
<span class="s1">bash-3.2$ docker rm mysite</span><br />
<span class="s1"></span><br />
<br />
<h3 class="p1">
<span class="s1">Install Docker Compose</span></h3>
<div class="p1">
<span class="s1"><a data-mce-href="https://github.com/docker/compose/releases" href="https://github.com/docker/compose/releases">https://github.com/docker/compose/releases</a></span></div>
<div class="p1">
<span class="s1">[root@biomos40 openstack]# curl -L <a href="https://www.blogger.com/null">https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname</a> -s`-`uname -m` > /usr/local/bin/docker-compose<br /> % Total % Received % Xferd Average Speed Time Time Time Current<br /> Dload Upload Total Spent Left Speed<br />100 600 0 600 0 0 1183 0 --:--:-- --:--:-- --:--:-- 1183<br />100 7739k 100 7739k 0 0 1392k 0 0:00:05 0:00:05 --:--:-- 1690k<br />[root@biomos40 openstack]# chmod +x /usr/local/bin/docker-compose</span></div>
<div class="p1">
<span class="s1">do command completion </span></div>
<pre><code class="hljs ruby">curl -<span class="hljs-constant">L</span> <span class="hljs-symbol">https:</span>/<span class="hljs-regexp">/github.com/docker</span><span class="hljs-regexp">/compose/releases</span><span class="hljs-regexp">/download/</span><span class="hljs-number">1.5</span>.<span class="hljs-number">2</span>/docker-compose-<span class="hljs-string">`uname -s`</span>-<span class="hljs-string">`uname -m`</span> > <span class="hljs-regexp">/usr/local</span><span class="hljs-regexp">/bin/docker</span>-compose</code></pre>
[openstack@biomos40 ~]$ docker-compose --version<br />
docker-compose version 1.5.2, build 7240ff3<br />
<br />
<br />
<pre></pre>
<div class="p1">
<span class="s1"><br /></span></div>
<h2 class="p1">
<span class="s1">Notes</span></h2>
<div class="p1">
<span class="s1"><a data-mce-href="http://developer.openstack.org/api-guide/quick-start/api-quick-start.html#client-intro" href="http://developer.openstack.org/api-guide/quick-start/api-quick-start.html#client-intro">http://developer.openstack.org/api-guide/quick-start/api-quick-start.html#client-intro</a></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<span class="s1"><br /></span>
<span class="s1">Links</span><br />
<span class="s1"><a href="https://docs.docker.com/articles/basics/">https://docs.docker.com/articles/basics/</a></span><br />
<span class="s1"><br /></span></div>
</div>
</div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com1tag:blogger.com,1999:blog-3810253236812062548.post-52792243372366398752015-03-25T18:53:00.001-07:002019-10-05T06:17:00.425-07:00JPA 2.0 Map Relationshipssee <a href="http://wiki.obrienlabs.cloud/display/DEV/Object+Relational+Mapping">http://wiki.obrienlabs.cloud/display/DEV/Object+Relational+Mapping</a><br />
I need a refreshment of JPA 2.0 support for Maps as I don't run into these type of relationships very often. The last time I went beyond Sets and Lists was when testing the Metamodel spec in 2010.<br />
<br />
The model used consists of a base subclass of a MappedSuperclass that has a unidirectional Map relationship keyed by Key entity to a map of Target entity values.<br />
<br />
<br />
<div class="p1">
@MappedSuperclass</div>
<div class="p2">
<span class="s1">public</span><span class="s2"> </span><span class="s1">class</span><span class="s2"> </span>BaseAbstract<span class="s2"> </span><span class="s3">{</span></div>
<div class="p3">
<br /></div>
<div class="p4">
<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="s2"><span class="Apple-tab-span"> </span></span>@Id</div>
<div class="p5">
<span class="s2"><span class="Apple-tab-span"> </span></span><span class="s4">@Column</span><span class="s3">(</span><span class="s5">name</span><span class="s3">=</span>"base_id"<span class="s3">)</span></div>
<div class="p2">
<span class="s2"><span class="Apple-tab-span"> </span></span><span class="s1">private</span><span class="s2"> </span>Long<span class="s2"> </span>ident<span class="s3">;</span></div>
<div class="p3">
<br /></div>
<div class="p6">
<span class="s2"><span class="Apple-tab-span"> </span></span><span class="s1">public</span><span class="s2"> </span><span class="s5">Long</span><span class="s2"> </span>getIdent()<span class="s2"> </span>{</div>
<div class="p7">
<span class="s2"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>return<span class="s2"> </span><span class="s5">ident</span><span class="s3">;</span></div>
<div class="p8">
<span class="Apple-tab-span"> </span><span class="s3">}</span></div>
<div class="p3">
<br /></div>
<div class="p6">
<span class="s2"><span class="Apple-tab-span"> </span></span><span class="s1">public</span><span class="s2"> </span><span class="s1">void</span><span class="s2"> </span>setIdent(<span class="s5">Long</span><span class="s2"> ident</span>)<span class="s2"> </span>{</div>
<div class="p8">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1">this</span><span class="s3">.</span><span class="s5">ident</span> <span class="s3">=</span> ident<span class="s3">;</span></div>
<div class="p8">
<span class="Apple-tab-span"> </span><span class="s3">}</span></div>
<div class="p4">
<span class="Apple-tab-span"> </span></div>
<br />
<div class="p6">
}</div>
<div class="p6">
<br /></div>
<div class="p1">
@Entity</div>
<div class="p2">
public<span class="s1"> </span>class<span class="s1"> </span><span class="s2">Base</span><span class="s1"> </span>extends<span class="s1"> </span><span class="s2">BaseAbstract</span><span class="s1"> </span><span class="s3">{</span></div>
<div class="p3">
<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span></span>@ManyToMany</div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="s4">private</span> <span class="s2">Map</span><span class="s3"><</span>Key<span class="s3">,</span> Target<span class="s3">></span> <span class="s2">targetMap</span><span class="s3">;</span></div>
<div class="p5">
<br /></div>
<div class="p6">
<span class="s1"><span class="Apple-tab-span"> </span></span><span class="s4">public</span><span class="s1"> </span><span class="s2">Map</span><<span class="s1">Key</span>,<span class="s1"> Target</span>><span class="s1"> </span>getTargetMap()<span class="s1"> </span>{</div>
<div class="p7">
<span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4">return</span><span class="s1"> </span>targetMap<span class="s3">;</span></div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="s3">}</span></div>
<div class="p5">
<br /></div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="s4">public</span> <span class="s4">void</span> <span class="s3">setTargetMap(</span><span class="s2">Map</span><span class="s3"><</span>Key<span class="s3">,</span> Target<span class="s3">></span> targetMap<span class="s3">)</span> <span class="s3">{</span></div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">this</span><span class="s3">.</span><span class="s2">targetMap</span> <span class="s3">=</span> targetMap<span class="s3">;</span></div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="s3">}</span></div>
<div class="p3">
<span class="Apple-tab-span"> </span></div>
<div class="p3">
<span class="Apple-tab-span"> </span></div>
<div class="p3">
<span class="Apple-tab-span"> </span></div>
<br />
<div class="p6">
}</div>
<div class="p6">
<br /></div>
<div class="p1">
@Entity</div>
<div class="p2">
public<span class="s1"> </span>class<span class="s1"> </span><span class="s2">Key</span><span class="s1"> </span><span class="s3">{</span></div>
<div class="p3">
<br /></div>
<div class="p4">
<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span></span>@Id</div>
<div class="p5">
<span class="s1"><span class="Apple-tab-span"> </span></span><span class="s4">@Column</span><span class="s3">(</span><span class="s2">name</span><span class="s3">=</span>"key_id"<span class="s3">)</span></div>
<div class="p6">
<span class="s1"><span class="Apple-tab-span"> </span></span><span class="s5">private</span><span class="s1"> </span>Long<span class="s1"> </span>ident<span class="s3">;</span></div>
<div class="p4">
<span class="Apple-tab-span"> </span></div>
<div class="p6">
</div>
<div class="p7">
}</div>
<div class="p6">
<br /></div>
<div class="p1">
@Entity</div>
<div class="p2">
public<span class="s1"> </span>class<span class="s1"> </span><span class="s2">Target</span><span class="s1"> </span><span class="s3">{</span></div>
<div class="p3">
<br /></div>
<div class="p4">
<span class="Apple-tab-span"> </span></div>
<div class="p1">
<span class="s1"><span class="Apple-tab-span"> </span></span>@Id</div>
<div class="p5">
<span class="s1"><span class="Apple-tab-span"> </span></span><span class="s4">@Column</span><span class="s3">(</span><span class="s2">name</span><span class="s3">=</span>"target_id"<span class="s3">)</span></div>
<div class="p6">
<span class="s1"><span class="Apple-tab-span"> </span></span><span class="s5">private</span><span class="s1"> </span>Long<span class="s1"> </span>ident<span class="s3">;</span></div>
<div class="p4">
<span class="Apple-tab-span"> </span></div>
<div class="p6">
</div>
<div class="p7">
}</div>
<div class="p7">
<br /></div>
<div class="p7">
DDL results</div>
<div class="p7">
<br /></div>
<div class="p1">
Hibernate: create table Base (base_id bigint not null, primary key (base_id)) type=InnoDB</div>
<div class="p1">
Hibernate: create table Base_Target (Base_base_id bigint not null, targetMap_target_id bigint not null, targetMap_KEY bigint not null, primary key (Base_base_id, targetMap_KEY)) type=InnoDB</div>
<div class="p1">
Hibernate: create table Key (key_id bigint not null, primary key (key_id)) type=InnoDB</div>
<div class="p1">
Hibernate: create table Target (target_id bigint not null, primary key (target_id)) type=InnoDB</div>
<div class="p1">
Hibernate: alter table Base_Target add index FKBFB1DEBFB32A10B2 (targetMap_KEY), add constraint FKBFB1DEBFB32A10B2 foreign key (targetMap_KEY) references Key (key_id)</div>
<div class="p1">
Hibernate: alter table Base_Target add index FKBFB1DEBF8408A9FE (targetMap_target_id), add constraint FKBFB1DEBF8408A9FE foreign key (targetMap_target_id) references Target (target_id)</div>
<div class="p1">
Hibernate: alter table Base_Target add index FKBFB1DEBF1B8E4DA4 (Base_base_id), add constraint FKBFB1DEBF1B8E4DA4 foreign key (Base_base_id) references Base (base_id)</div>
<div class="p1">
Hibernate: alter table Base_Target drop foreign key FKBFB1DEBFB32A10B2</div>
<div class="p1">
Hibernate: alter table Base_Target drop foreign key FKBFB1DEBF8408A9FE</div>
<div class="p7">
</div>
<div class="p1">
Hibernate: alter table Base_Target drop foreign key FKBFB1DEBF1B8E4DA4</div>
<div class="p1">
<br /></div>
<div class="p1">
Sorry EclipseLink but Hibernate is used in almost all projects not directly running on WebLogic or the RI version of GlassFish.</div>
<div class="p1">
<br /></div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com1tag:blogger.com,1999:blog-3810253236812062548.post-43125259794370091792015-03-21T22:44:00.003-07:002016-02-02T20:05:18.692-08:00Haswell Intel i7 5820K Overclock PC BuildIn progress...<br />
Even though I have a recent love affair with Apple since 2011 - especially with their top of the line MacBook Pro 15 inch 2.6 GHz 4960HQ - I still need more parallel processing muscle. Amazon offers the 18 core xeon C4-8xLarge at a spot price of $0.28 for pure Java console processing jobs - but a real physical machine is good for Oculus Rift for example.<br />
<br />
I have decided not to buy anymore commercial machines with low end parts and build them myself. This article is a summary of my first build.<br />
<br />
Parts: I am leaning towards Asus, Corsair and Samsung parts.<br />
<br />
Intel Haswell i7-5820K<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiiRw7Tao9B6MQ8JGjtuXDXohcQYC4c-PZhhiL2keP4LBafP4aS5LGr8QHZaHdKt4R34GPlrf00C4MbdjVm4SmxvuUXWZbW_5ymXV46D_v2bip3KaL1gL0jlNzsfcXjuizCbqIYVSahVR9/s1600/2015-02-26+19.11.26.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiiRw7Tao9B6MQ8JGjtuXDXohcQYC4c-PZhhiL2keP4LBafP4aS5LGr8QHZaHdKt4R34GPlrf00C4MbdjVm4SmxvuUXWZbW_5ymXV46D_v2bip3KaL1gL0jlNzsfcXjuizCbqIYVSahVR9/s1600/2015-02-26+19.11.26.jpg" width="640" /></a></div>
<br />
Corsair liquid cooler<br />
Corsair case<br />
Corsair power supply<br />
Corsair Vengeance LPX 32GB 2 x (4x8GB) DDR4 2666MHz CL16 Quad-Channel<br />
Asus X99 motherboard<br />
Intel SSD<br />
Samsung SSD<br />
<br />
A Flir heatmap of the i7 5820, the DDR4 ram and the first GTX-970 before I added the 2nd Asus Strix card<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6yHPuiiNUkchmqySPG_P_sFfSegmfoKAzIJUCKVfWpaLDTWYbVkQpMs61GQZ_ZlP8ZyJQTGRY4fBEHwgQXdwxuE3xUF-28WQQSAghGSRn4uAoNYPqj5cMS7Zw2U129EUK_9SfRzob15QD/s1600/2015-03-17+07.50.19.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6yHPuiiNUkchmqySPG_P_sFfSegmfoKAzIJUCKVfWpaLDTWYbVkQpMs61GQZ_ZlP8ZyJQTGRY4fBEHwgQXdwxuE3xUF-28WQQSAghGSRn4uAoNYPqj5cMS7Zw2U129EUK_9SfRzob15QD/s1600/2015-03-17+07.50.19.jpg" width="640" /></a></div>
The case after running both GTX-970 cards under load<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhDVMz4lqoZkumu28J8fgCU-jha7-H3rWc9AsVbpohvIjOrj8FTeCXdVAiw31zzrKsmO2Ng3pAjKUKzUHxMS3MtayrnEFakC1_C-_4-jFdWpQ-GL4BbucFIGJ1Vwe-6Upm1-H4Ah3sWijn/s1600/2015-03-25+21.04.00_flir_case.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhDVMz4lqoZkumu28J8fgCU-jha7-H3rWc9AsVbpohvIjOrj8FTeCXdVAiw31zzrKsmO2Ng3pAjKUKzUHxMS3MtayrnEFakC1_C-_4-jFdWpQ-GL4BbucFIGJ1Vwe-6Upm1-H4Ah3sWijn/s1600/2015-03-25+21.04.00_flir_case.jpg" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTfIb7P9nWmJPwLZ87z3t1z8KgWjgKQkNvwgOS44Bju06YX1UtidEaIlkcVBdkadOTyLqvQi7j6CbSV-eHyV3mbPnKiRq2d0b2JCVifMnO9omlCAvtCgeXYfkdsPoR2HAA6otj8aJMLikP/s1600/2015-02-26+19.36.10.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTfIb7P9nWmJPwLZ87z3t1z8KgWjgKQkNvwgOS44Bju06YX1UtidEaIlkcVBdkadOTyLqvQi7j6CbSV-eHyV3mbPnKiRq2d0b2JCVifMnO9omlCAvtCgeXYfkdsPoR2HAA6otj8aJMLikP/s1600/2015-02-26+19.36.10.jpg" width="640" /></a></div>
Corsair case with 32G Corsair DDR4, i7-5820K, Corsair 100 cooler, Corsair 860i PSU and 2 Asus Strix GTX-970 in SLI<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxSdMgJx2gJ023VxpZ1IMFi_QOenFWofxQv7u9iNTxQsCNavWH7WyC32iZ_-vGnu-IhMJbdaIlPKUjoYa1TQj7xS9fstHbZQDWDgRZjWIq09VaTgbDY7y2yhzqVauWhf0Vhb753eGQ4Wn4/s1600/IMG_0031_5820_sld_970_32g_pc.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxSdMgJx2gJ023VxpZ1IMFi_QOenFWofxQv7u9iNTxQsCNavWH7WyC32iZ_-vGnu-IhMJbdaIlPKUjoYa1TQj7xS9fstHbZQDWDgRZjWIq09VaTgbDY7y2yhzqVauWhf0Vhb753eGQ4Wn4/s1600/IMG_0031_5820_sld_970_32g_pc.JPG" width="640" /></a></div>
<br />
Details:<br />
The top i7-5960X is just too expensive at this point - with the recent drop in oil it comes out to almost $1500 just for the chip. If you don't mind a 100MHz drop and don't plan on running 3 GPU cards then the 5820 can be substituted for the 5930 at a savings of $300<br />
<br />
I accidentally scratched off about 5% of the factory paste. I went ahead anyway instead of redoing it with a rice ball sized drop. This may be why a couple of my six cores show a 10 degree difference under load.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXXtdekcDCLo62rFcNplJJ2Y_L1HTGlosi39FAU7N1fr92GRSIg5ZaRu2r-MBJOHgxHsknSOXUVkRe9MYq9mbSJDxVyAJdgCRB-Ka8oxh0EyOAVwIOyw66A9MXrl8Uc79TrHb-HmvProIm/s1600/2015-02-26+19.07.52+HDR.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXXtdekcDCLo62rFcNplJJ2Y_L1HTGlosi39FAU7N1fr92GRSIg5ZaRu2r-MBJOHgxHsknSOXUVkRe9MYq9mbSJDxVyAJdgCRB-Ka8oxh0EyOAVwIOyw66A9MXrl8Uc79TrHb-HmvProIm/s1600/2015-02-26+19.07.52+HDR.jpg" width="480" /></a></div>
<br />
<br />
Dual ASUS Strix GTX-970 cards in SLI<br />
I know that the 5820 because it only has 28 PCIx lanes - supports an SLI config of 16x/8x instead of 16x/16x. However until I get $1500 to get a 5960X - I will run one of the cards at 8x.<br />
Interesting that my 2nd card purchased about 2 weeks later from the same supplier has a flipped Strix logo plate - I don't know which is the right orientation. Adding the 2nd Strix GTX-970 was pretty easy in the large Corsair case. Running the dual GPU's ups the wattage from an idle of 240W to 505W under 17% CPU load. Note: running the CPU at 100% as well pushed the box to 700W which is pushing my 860W Corsair PSU.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQQh6hlnUtKdrdck8T5lqIcMFn4N2P6iCklcUbuzqoEtGDA1SqDlg4Itf2NYzAF3nksSQuKVEZxyhXzDuDRqwpOKDKiC4y3w8pPNrNbccAvwG5UYM9tFrirMukzQtt6VUkifZuio1VGc2G/s1600/IMG_0027_strix_sli_mixed_logos.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQQh6hlnUtKdrdck8T5lqIcMFn4N2P6iCklcUbuzqoEtGDA1SqDlg4Itf2NYzAF3nksSQuKVEZxyhXzDuDRqwpOKDKiC4y3w8pPNrNbccAvwG5UYM9tFrirMukzQtt6VUkifZuio1VGc2G/s1600/IMG_0027_strix_sli_mixed_logos.JPG" width="480" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
The Flir heatmap of the two SLI GTX-970 cards with their inverted logos</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7IdM1SJVnQjQzS9Tb6STiPmatslWRnXj2e0H__d2qlCE2Lei8x28903jqhKL89kS0DVg9vl1RhNcQzFXJR5n4VjGVq-30uvNQMKhj50kGWOZrD0hfFRUvJC1Xe5hQ7lIk62Y3tlD8Uatl/s1600/2015-03-25+21.07.17_sli_strix_heatmap.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7IdM1SJVnQjQzS9Tb6STiPmatslWRnXj2e0H__d2qlCE2Lei8x28903jqhKL89kS0DVg9vl1RhNcQzFXJR5n4VjGVq-30uvNQMKhj50kGWOZrD0hfFRUvJC1Xe5hQ7lIk62Y3tlD8Uatl/s1600/2015-03-25+21.07.17_sli_strix_heatmap.jpg" width="480" /></a></div>
<br />
<br />
Results:<br />
After conservative overclocking to 4.2GHz using a 125MHz base clock from 100 and a 33x multiplier I see a 50% speed increase over my fastest Mac i7-4960<br />
<br />
My Oculus Rift headset runs fine on this 5820 machine - it ran fine on my MacBook Pro 4960 as well as my Asus ROG 3610.<br />
<br />
Need to run faster than 4.1GHz <br />
<a href="http://www.overclockers.com/forums/showthread.php/754365-Asus-X99-A-CPU-voltage-settings-need-an-experienced-user">http://www.overclockers.com/forums/showthread.php/754365-Asus-X99-A-CPU-voltage-settings-need-an-experienced-user</a><br />
<br />
Intel 750 NVMe to 4 x 256G Samsung 850pro raid0 runs at 8Gbps<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_3ebkbiKkLlZX7YhcD3BInn9JbdrnXs_eSrNllmix32ZN_SDazprNk-x3nochyI98r42zF4UaqmhUkybnU4d7758Jw1LYbdeUuG7uIUP7-71YUDOKExm5udkNQbWtAFG6zw3O187b8DCZ/s1600/8gbps_intel750NVMe_to_4x256Samsung850pro_raid0.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_3ebkbiKkLlZX7YhcD3BInn9JbdrnXs_eSrNllmix32ZN_SDazprNk-x3nochyI98r42zF4UaqmhUkybnU4d7758Jw1LYbdeUuG7uIUP7-71YUDOKExm5udkNQbWtAFG6zw3O187b8DCZ/s1600/8gbps_intel750NVMe_to_4x256Samsung850pro_raid0.png" /></a></div>
<br />Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com1tag:blogger.com,1999:blog-3810253236812062548.post-47284244822774122822015-02-14T21:25:00.002-08:002015-02-20T09:10:31.371-08:00AWS Spot EC2 Instances for Elastic Beanstalk saves around 70-90% of instance costs over reserved or on-demand Amazon AWS is an excellent service - that Amazon runs very well. There are opportunities to save between 25 and 60% - but there is and even better way to reduce costs for non-production development servers.<br />
On-demand instance cost for AWS resources can be reduced by 25% using no-upfront reservations or by 56% by purchasing the instance for 3 years. However the use of the spot instance market allows a reduction from 80 to 90% - but not in beanstalk.<br />
There is however a way to reserve spot instances for elastic beanstalk through the .ebextension mechanism like we do for other types of EC2 instance customization. There are the usual problems of not being compatible with auto-scaling but this price reduction is excellent for development instances.<br />
Enter the following .config with a spot price that is just above the lowest price across all regions for the past 3 months - so you run the lowest risk of loosing the instance. For example I reserved the new C4.Large at 0.018 which is above the normal spot of 0.0161 - however this is a lot better than the on-demand price of 0.160 - or 10% of the normal price.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK-TWLKnB-zER6hcysRZmbwPa3a3Dnfpa8zwGtXytfcQN-07aWQM0aSWZrzrC5uEHfN-8p-YnMe_fuVqx-gOGDQU9Rzq8LJi3ZDr9h1HhQfUr-w-ozuqGhz8bCShyKAaD2o4nGsKJlpMvQ/s1600/Screenshot+2015-02-15+00.19.29.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK-TWLKnB-zER6hcysRZmbwPa3a3Dnfpa8zwGtXytfcQN-07aWQM0aSWZrzrC5uEHfN-8p-YnMe_fuVqx-gOGDQU9Rzq8LJi3ZDr9h1HhQfUr-w-ozuqGhz8bCShyKAaD2o4nGsKJlpMvQ/s1600/Screenshot+2015-02-15+00.19.29.png" height="488" width="640" /></a></div>
Note the spot price which is a reduction of 90% on the AWS EC2 on-demand price. We will put this into the .ebextension directory in a config file<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3C-_MWGGqc0MGldTXjccsLSSx1WxvVI9xI2YXChPwo9IxQTO63dhEQiBMZwxFY6PGswvCOVpeIL8Filo4xJx9s1TXep31oq998wKSaLpHQj9Aim8CoS0JlrDGc0hHIsnwwPWvJGD_zG0H/s1600/Screenshot+2015-02-14+23.55.59.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3C-_MWGGqc0MGldTXjccsLSSx1WxvVI9xI2YXChPwo9IxQTO63dhEQiBMZwxFY6PGswvCOVpeIL8Filo4xJx9s1TXep31oq998wKSaLpHQj9Aim8CoS0JlrDGc0hHIsnwwPWvJGD_zG0H/s1600/Screenshot+2015-02-14+23.55.59.png" height="163" width="640" /></a></div>
<br />
Add the following to your pom.xml if you are using "ebextensions" instead of the target ".ebextensions"<br />
<br />
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <build></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <plugins></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <plugin></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <groupId>org.apache.maven.plugins</groupId></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <artifactId><span style="text-decoration: underline;">maven</span>-war-<span style="text-decoration: underline;">plugin</span></artifactId></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <configuration></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <webResources></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <resource></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <directory>src/main/ebextensions</directory></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <targetPath>.ebextensions</targetPath></span></div>
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> <filtering>true</filtering> </span></div>
<br />
<div style="font-family: Monaco; font-size: 11px;">
<span style="color: #38761d;"> </resource></span></div>
<div style="color: #e6e8e9; font-family: Monaco; font-size: 11px;">
<br /></div>
After you build your WAR and deploy it, you will initially see some scaling errors until the spot request is fulfilled.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXSaRkcu6tRrZ1EYxfch2YVAG7OPBdx-401WMd-f1-rKCDW7hSTN3ebQwhzk58ixGLrAGCM8FqCSfLABSVxIKbANaQEhS61lqsoeIhQ3v7xla6m5iP_ik4Cfkghhk1PDfDBn45BFC3RTfe/s1600/Screenshot+2015-02-15+00.01.49.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXSaRkcu6tRrZ1EYxfch2YVAG7OPBdx-401WMd-f1-rKCDW7hSTN3ebQwhzk58ixGLrAGCM8FqCSfLABSVxIKbANaQEhS61lqsoeIhQ3v7xla6m5iP_ik4Cfkghhk1PDfDBn45BFC3RTfe/s1600/Screenshot+2015-02-15+00.01.49.png" height="96" width="640" /></a></div>
<br />
As soon as your instance is up and the war completes deployment - the healthcheck will report a good instance.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnxH1hgVf7k2O-uXeqlBOoyYp3bLbrUUkAviJIkxeDeGzYa6edcgNKa6pVnMDEl3oe0JtHZsAjspTBJiDFYIyPoEySGqGoD-v0oRSWkPAh-B2B35st4mgodQsfDLn6DP0oUd1oaeND4IoI/s1600/Screenshot+2015-02-15+00.03.08.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnxH1hgVf7k2O-uXeqlBOoyYp3bLbrUUkAviJIkxeDeGzYa6edcgNKa6pVnMDEl3oe0JtHZsAjspTBJiDFYIyPoEySGqGoD-v0oRSWkPAh-B2B35st4mgodQsfDLn6DP0oUd1oaeND4IoI/s1600/Screenshot+2015-02-15+00.03.08.png" height="304" width="640" /></a></div>
<br />
If you are running a set of development servers like 10 M3.xlarge instances at a on-demand price of $0.28/hour - this comes out to $67/day or $25,000/year. Switching to using spot instances at the current stable price in us-east-1b of $0.0321 - but lets use $0.034 - <b><span style="color: #cc0000;">would come out to $8/day or $3000/year - a cost savings of $22,000 at 88% off. Actual savings would be more in line with $8,000 or 72% if you normally use 3yr reserved instances.</span></b><br />
<br />
Note that you can loose your instance at any time if the lowest spot price jumps.<br />
There are ways to get environment variables into the config files without resorting to commands or sh scripts - so we can change the spot price when we want to up the instance to a C4.8xlarge for example.<br />
<br />
/michael<br />
<br />
<blockquote class="twitter-tweet" lang="en">
Use .ebextensions to save 90% using spot price <a href="https://twitter.com/hashtag/EC2?src=hash">#EC2</a> instances for <a href="https://twitter.com/hashtag/AWS?src=hash">#AWS</a> <a href="https://twitter.com/aws_eb">@aws_eb</a> elastic beanstalk <a href="https://twitter.com/awscloud">@awscloud</a> deploys
<a href="http://t.co/HoU9LOGXOq">http://t.co/HoU9LOGXOq</a><br />
— Michael_OBrien (@_mikeobrien) <a href="https://twitter.com/_mikeobrien/status/566837646144446464">February 15, 2015</a></blockquote>
<script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script>
<br/>
<blockquote class="twitter-tweet" lang="en"><p>Presenting on <a href="https://twitter.com/awscloud">@awscloud</a> Elastic Beanstalk <a href="https://twitter.com/aws_eb">@aws_eb</a> in a VPC for <a href="https://twitter.com/hashtag/AWS?src=hash">#AWS</a> Ottawa Users Group - Feb 24 at <a href="https://twitter.com/bitHeads">@bitHeads</a> Ottawa - <a href="http://t.co/YsH2dTwLUu">http://t.co/YsH2dTwLUu</a></p>— Michael_OBrien (@_mikeobrien) <a href="https://twitter.com/_mikeobrien/status/568818341126623232">February 20, 2015</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com4tag:blogger.com,1999:blog-3810253236812062548.post-60254896798711153482015-01-12T18:41:00.000-08:002015-02-25T07:32:04.067-08:00Amazon Web Services C4 Haswell 36 core compute instances are ready Back in Nov 2014 the announcement of the new Haswell chips designed and implemented by Intel specifically for AWS EC2 was very exciting. Today the 36 thread <b>(18 physical cores)</b> <span style="color: #333333; font-family: HelveticaNeue, Helvetica, Helvetica, Arial, sans-serif; font-size: 14px; line-height: 22.3999996185303px;">Xeon E5-2666 v3 </span>processor @ 2.9 Ghz (an improvement over the 30 core Ivy Bridge predecessor) arrived. Where else can you test an <a href="http://aws.amazon.com/ec2/purchasing-options/reserved-instances/">$18000</a> computer by renting it for $1.80/hr.<br />
<a href="https://aws.amazon.com/blogs/aws/now-available-new-c4-instances/">https://aws.amazon.com/blogs/aws/now-available-new-c4-instances/</a><br />
<br />
I put the new EC2 instance running on the top of the line C4-8xLarge processor to a concurrency benchmark written in Java that is designed to test various levels of multithreading load by computing the collatz series. No tests on HD, network or memory throughput were tested - the benchmark is purely integer computation with no optimization for the new haswell vector instructions. As a comparison I ran the same parallel processing benchmark on the g2-2xlarge, r3-8xlarge and older c3-8xlarge instances.<br />
<div class="p1">
<br />
<span style="color: #6aa84f;"><b>I see around 11% performance increase using the C4 over the C3.</b></span><br />
<br />
Time performance for 64 threads running various levels of 2^22 to 1 concurrent work packets - in ms where a lower number is better.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAADC0011uZufKelqCfQ0otTaSK1qPQ422bgeI-atUMUh1TZXJV9knxzoHmraQquh1LwF98RJUncViM46Ggfs-g3vg8ZoX2LZ-zElrSPoZbifLhFd14ZMKqhUmr2lroTkb1GODSfR0D2j1/s1600/c4_8xlarge_parallel_perf_64threads.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAADC0011uZufKelqCfQ0otTaSK1qPQ422bgeI-atUMUh1TZXJV9knxzoHmraQquh1LwF98RJUncViM46Ggfs-g3vg8ZoX2LZ-zElrSPoZbifLhFd14ZMKqhUmr2lroTkb1GODSfR0D2j1/s1600/c4_8xlarge_parallel_perf_64threads.png" height="468" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
This is the same diagram with the lower concurrency levels of 1 to 2 removed.</div>
<div class="separator" style="clear: both; text-align: center;">
As you can see the blue line for the C4 is giving us the expected performance increase for the core increase from 32 to 36 cores.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6rmxS-m2zEQNCNWINrFBN7opGKHMKc-9WobmWXuRaWWlR4UVmjXly4hu0P6XoROo-RRwm3bE_kyd8AKJXjPxRk56Ut868ePTIr32KVSG6VeAGiypHeJimahm3yMS42jl0nQGbQ-RqQv7W/s1600/c4_8xlarge_parallel_perf_64threads_zoom.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6rmxS-m2zEQNCNWINrFBN7opGKHMKc-9WobmWXuRaWWlR4UVmjXly4hu0P6XoROo-RRwm3bE_kyd8AKJXjPxRk56Ut868ePTIr32KVSG6VeAGiypHeJimahm3yMS42jl0nQGbQ-RqQv7W/s1600/c4_8xlarge_parallel_perf_64threads_zoom.png" height="498" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Notice that the 3 upper lines are from 2 physical machine processors and the GPU VM - here the 4960 on a Macbook Pro, the 3610 on an Asus ROG and the g2-2xlarge cloud instance. The varying 3 lines at the bottom of the graph for the r3, c3 and c4 instances are virtual cores that are subject to burst and adjacent loads.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid2DCs5sLI-zEO_as5rluYOUyjicSTMotqOplOiCvT3wzi631JHbMsu2WxWgZpveuEB-Nhs88mebG8EjXSNQy1ZnzewSpENwIIXh-0v_uP1urz1zGaSaPaeLoxDDMA0A_5C6x0yuxFYOj_/s1600/c4_ec2_cloudwatch.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid2DCs5sLI-zEO_as5rluYOUyjicSTMotqOplOiCvT3wzi631JHbMsu2WxWgZpveuEB-Nhs88mebG8EjXSNQy1ZnzewSpENwIIXh-0v_uP1urz1zGaSaPaeLoxDDMA0A_5C6x0yuxFYOj_/s1600/c4_ec2_cloudwatch.png" height="624" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
The following is the concurrency map for various levels of threading from 1 to 64 across various levels of concurrent work packets. As you can see the processor really shines when we feed it over 1k work items distributed over at least 64 threads.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguVzMchlo0E7i6s11C52he9V9v7G5MGLVFy7jC6Q1YQtdyMyuW2wciQv1ei-mKUcdolkHw2LRJtNfBKChXi30DKgiAecA2Z6EeS7dqU8sXGumqgKkAP-5og7EIMnv6LhcmxaRwNKVQ-xnj/s1600/c4_8xlarge_parallel_perf_surface.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguVzMchlo0E7i6s11C52he9V9v7G5MGLVFy7jC6Q1YQtdyMyuW2wciQv1ei-mKUcdolkHw2LRJtNfBKChXi30DKgiAecA2Z6EeS7dqU8sXGumqgKkAP-5og7EIMnv6LhcmxaRwNKVQ-xnj/s1600/c4_8xlarge_parallel_perf_surface.png" height="422" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Note: the bumps in the graph occur when the OS scheduled work that may affect the test 2-5%. More variance happens at the virtualization layer where burst mode may occur on or beside our processor slice.<br />
The level of performance increase looks good - even though we only moved from 2.8 to 2.9 Ghz - we are using a more efficient processor architecture on top of getting 4 more cores. The single core performance of the C4 is not very good (just like all the other R3, C3 and G2 instances) - as expected but it excels is heavy concurrent workloads.<br />
What is particularly exciting is the 3400% processor usage in the "top" command below.<br />
<br />
The concurrency test varies the thread pool from 1 to 1024 threads in powers of 2. The optimum number of threads is usually around 2 times the # of threaded processors - in this case 64<br />
<div class="p1">
<span class="s1"><i>Cpu(s): 39.2%us, 56.9%sy, 0.0%ni, 4.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</i></span></div>
<div class="p1">
<span class="s1"><i>Mem: 61847440k total, 6289296k used, 55558144k free, 23156k buffers</i></span></div>
<div class="p1">
<span class="s1"><i>Swap: 0k total, 0k used, 0k free, 424720k cached</i></span></div>
<div class="p3">
<span class="s1"><i> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND </i></span></div>
<br />
<div class="p1">
<span class="s1"><i>55371 ec2-user 20 0 34.1g 5.0g 10m S <b>3427.6 </b> 8.6 118:44.18 java </i></span></div>
<div class="p1">
-</div>
</div>
<div class="p1">
Disclaimers<br />
- I forgot to convert ms to sec - sorry<br />
- This benchmark is a simple fork-join MapReduce test that may be affected by the other 2-5% OS processes occurring in the background - hey it has only been less than 24 hours since the processor went live.<br />
- No really useful work was done during the actual benchmark tests - no collatz number was found past 2^60 yet!<br />
- Note the fastest time in this run was 130 sec for the C4 but I have seen a performance spike even on the R3 instances at 105 sec.<br />
- Use spot instances to save on the cost by up to 85%.<br />
<a href="http://eclipsejpa.blogspot.ca/2015/02/aws-spot-instances-for-elastic.html">http://eclipsejpa.blogspot.ca/2015/02/aws-spot-instances-for-elastic.html</a><br />
<br />
<blockquote class="twitter-tweet" lang="en">
New <a href="https://twitter.com/awscloud">@awscloud</a> 36-core <a href="https://twitter.com/intel">@Intel</a> Xeon E5-2666 v3 c4-8xlarge <a href="https://twitter.com/hashtag/EC2?src=hash">#EC2</a> concurrency throughput results
<a href="http://t.co/CNbU9vge6v">http://t.co/CNbU9vge6v</a> <a href="http://t.co/BZXM9OY3oc">pic.twitter.com/BZXM9OY3oc</a><br />
— Michael_OBrien (@_mikeobrien) <a href="https://twitter.com/_mikeobrien/status/554847442785816576">January 13, 2015</a></blockquote>
<script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script>
</div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com0tag:blogger.com,1999:blog-3810253236812062548.post-50650873371160960652014-12-19T07:49:00.001-08:002015-06-04T13:04:13.020-07:00Amazon Web Services Developer Flight Checklist<br />
Working in the clouds on Amazon AWS benefits from some best practices. I am capturing details into a pre-flight checklist before you fly between the clouds as I come across them in my daily development.<br />
<br />
<h2>
Amazon Web Service Best Practices</h2>
<h3>
Elastic Beanstalk</h3>
<div>
1) Add JVM tracking like New Relic</div>
<div>
2) Add Log tracking like SumoLogic or Splunk</div>
<div>
3) Add GEOIP tracking like MaxMind geoIP</div>
<div>
When deploying the mod_geoip conf file to elastic beanstalk remember that we are running behind an elastic load balancer proxy. This proxy will add its IP address to the <span style="background-color: white; color: #333333; font-family: 'Courier New', Courier, monospace; font-size: 14px; line-height: 19.6000003814697px; text-align: justify;">%{X-Forwarded-For}</span> variable. geoIP will not be able to pick out the actual client IP until you add the following parameter to the conf file to workaround this nonstandard <a href="http://publib.boulder.ibm.com/tividd/td/ITWSA/ITWSA_info45/en_US/HTML/guide/c-logs.html#ncsa">NCSA</a> log format.<br />
see <a href="http://dev.maxmind.com/geoip/legacy/mod_geoip2/#Proxy-Related_Directives">http://dev.maxmind.com/geoip/legacy/mod_geoip2/#Proxy-Related_Directives</a></div>
<div>
4) Install an SSL certificate on the ELB (Elastic Load Balancer)<br />
Follow <a href="http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/ssl-server-cert.html">http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/ssl-server-cert.html</a> but gunzip, tar xvf and cd to the install directory first.<br />
<br />
EC2 instances need the Java AWS SDK<br />
<div class="p1">
<span class="s1">obrienlabs-mbp15:_deployment michaelobrien$ sudo scp ~/.aws/credentials ec2-user@54.5.6.7:/home/ec2-user</span></div>
<div class="p1">
<span class="s1">Password:</span></div>
<br />
<div class="p1">
<span class="s1">credentials 100% 221 0.2KB/s 00:00 </span></div>
</div>
<div>
<br /></div>
Pick your JVM<br />
<span style="font-family: monospace; font-size: 13px;">They do seem to quietly upgrade the JDK after official published versions - they have since went for 1.6.0_24 to 32 and 1.7.0_51 to 72</span><br />
<span style="font-family: monospace; font-size: 13px;">OpenJDK versions lag SUN by about a week, but amazon seems to favour Java 8</span><br />
<span style="font-family: monospace; font-size: 13px;">The 3 ami versions for 6, 7 and 8 look to be behind more as we drop JDK versions - with Java 8 up to date and 6 way behind.</span><br />
<br style="font-family: monospace; font-size: 13px;" />
<span style="font-family: monospace; font-size: 13px;">OpenJDK versions</span><br />
<span style="font-family: monospace; font-size: 13px;">Beanstalk Tomcat 8 Java 8 = 1.8.0_25 (latest 25) 2014.09 v1.0.0</span><br />
<span style="font-family: monospace; font-size: 13px;">Beanstalk Tomcat 7 Java 7 = 1.7.0_65 (latest 72) 2014.09 v1.0.9</span><br />
<span style="font-family: monospace; font-size: 13px;">Beanstalk Tomcat 7 Java 6 = 1.6.0_32 (latest 85) 2014.09 v1.0.9</span><br />
<br style="font-family: monospace; font-size: 13px;" />
<span style="font-family: monospace; font-size: 13px;">Their official page on supported platforms</span><br />
<a href="https://anywhere.exchserver.com/owa/redir.aspx?C=KX_za7-sokSIt1aRDffLfGBhl1jTC9JIIUEuy3sFWXYfQaDC0KupJWxf--cIqT2ssd5myID5wXw.&URL=http%3a%2f%2fdocs.aws.amazon.com%2felasticbeanstalk%2flatest%2fdg%2fconcepts.platforms.html%23concepts.platforms.java" style="font-family: monospace; font-size: 13px;" target="_blank">http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.platforms.html#concepts.platforms.java</a><br />
On all your windows machines install cygwin and enable the ssh server.<br />
<a href="http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/">http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/</a><br />
<br />
<h2>
AWS Technical Details</h2>
<h3>
Amazon Service Disruptions:</h3>
Some AWS services encounter issues from time to time at the service, zone or region. If you happen to be in the console you see the occasional message at the top - however the RSS feed gives a bit more detail on the issue. Again if you build in standard zone and region redundancy then failover kicks in and no service disruption affects your instances.<br />
<br />
Install the client - passed by my teamlead<br />
https://chrome.google.com/webstore/detail/rss-feed-reader/pnjaodmkngahhkoihejjehlcdlnohgmp?hl=en<br />
<br />
VPC restrictions<br />
<br />
<div class="MsoNormal">
<span style="color: #1f497d; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-theme-font: minor-bidi; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">us-east-1a is restricted by amazon for new subnets (I read this
somewhere – and also read that is was full) – and for t2 instances<o:p></o:p></span></div>
<br />
<div class="MsoNormal">
<span style="color: #1f497d; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-theme-font: minor-bidi; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">us-east-1c came back as restricted during a CloudFormation json deployment run – but I read
somewhere that this may be only for customers that have accounts predating when
the “default” VPC (only 1 public subnet at 172.31.0.0/16) – customers like my personal that signed up after Aug 2014 get are in the default VPC.<o:p></o:p></span></div>
<h3>
Elastic Beanstalk</h3>
<h4>
Configuring a local OSX dev environment for Beanstalk</h4>
<div>
see</div>
<div>
<a href="https://news.ycombinator.com/item?id=7172060">https://news.ycombinator.com/item?id=7172060</a></div>
<div>
<br />
<h4>
Get an SSL Certificate signed by Entrust</h4>
<div class="p1">
<a href="http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https.html">http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https.html</a></div>
<div class="p2">
Entrust will reply to your request within 10 min.<br />
<br /></div>
<div class="p3">
<span class="s1">obrienlabs-mbp15:nutridat_domain_cert michaelobrien$ openssl genrsa 2048 > privatekey.pem</span></div>
<div class="p3">
<span class="s1">Generating RSA private key, 2048 bit long modulus</span></div>
<div class="p3">
<span class="s1">..+++</span></div>
<div class="p3">
<span class="s1">.............+++</span></div>
<div class="p3">
<span class="s1">e is 65537 (0x10001)</span></div>
<div class="p4">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1">You are about to be asked to enter information that will be incorporated</span></div>
<div class="p3">
<span class="s1">into your certificate request.</span></div>
<div class="p3">
<span class="s1">What you are about to enter is what is called a Distinguished Name or a DN.</span></div>
<div class="p3">
<span class="s1">There are quite a few fields but you can leave some blank</span></div>
<div class="p3">
<span class="s1">For some fields there will be a default value,</span></div>
<div class="p3">
<span class="s1">If you enter '.', the field will be left blank.</span></div>
<div class="p3">
<span class="s1">-----</span></div>
<div class="p3">
<span class="s1">Country Name (2 letter code) [AU]:CA</span></div>
<div class="p3">
<span class="s1">State or Province Name (full name) [Some-State]:Ontario</span></div>
<div class="p3">
<span class="s1">Locality Name (eg, city) []:Ottawa</span></div>
<div class="p3">
<span class="s1">Organization Name (eg, company) [Internet Widgits Pty Ltd]:.</span></div>
<div class="p3">
<span class="s1">Organizational Unit Name (eg, section) []:.</span></div>
<div class="p3">
<span class="s1">Common Name (e.g. server FQDN or YOUR name) []:nutridat.org</span></div>
<div class="p3">
<span class="s1">Email Address []:michael@o</span></div>
<div class="p4">
<span class="s1"></span><br /></div>
<div class="p3">
<span class="s1">Please enter the following 'extra' attributes</span></div>
<div class="p3">
<span class="s1">to be sent with your certificate request</span></div>
<div class="p3">
<span class="s1">A challenge password []:</span></div>
<br />
<div class="p3">
<span class="s1">An optional company name []:.</span></div>
<div class="p3">
<span class="s1"><br /></span></div>
<br /></div>
<div>
Elastic Beanstalk on AWS runs on a customized EC2 AMI as Tomcat 6/7/8 on top of Apache HTTPD.<br />
Configure your maven project to compile against Tomcat 7 or 8<br />
<br />
<pre class="brush:java"> <dependencies>
<dependency>
<groupid>junit</groupid>
<artifactid>junit</artifactid>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<!-- Provided by container -->
<dependency>
<groupid>jstl</groupid>
<artifactid>jstl</artifactid>
<version>1.2</version>
<!-- not provided in our container -->
</dependency>
<dependency>
<groupid>javax.el</groupid>
<artifactid>el-api</artifactid>
<version>2.2</version>
<scope>provided</scope>
</dependency>
<!-- dependency>
<groupid>javax.servlet</groupId>
<artifactid>javax.servlet-api</artifactId>
<version>3.0.1</version>
<scope>provided</scope>
</dependency-->
<dependency>
<groupid>org.apache.tomcat</groupid>
<artifactid>tomcat-el-api</artifactid>
<version>${version.tomcat}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupid>org.apache.tomcat</groupid>
<artifactid>tomcat-servlet-api</artifactid>
<version>${version.tomcat}</version>
<scope>provided</scope>
</dependency>
</dependencies>
</pre>
<br />
No automatic maven dependencies in eclipse/sts? add the following<br />
<pre class="brush:java"> <classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER">
<attributes>
</attributes></classpathentry></pre>
<pre class="brush:java"><classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"><attributes><attribute name="maven.pomderived" value="true">Get the AWS CLI and EC2 CLI</attribute></attributes></classpathentry></pre>
<pre class="brush:java"><classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"><attributes><attribute name="maven.pomderived" value="true"><a href="http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/set-up-ec2-cli-linux.html#setting_up_ec2_command_linux">http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/set-up-ec2-cli-linux.html#setting_up_ec2_command_linux</a></attribute></attributes></classpathentry></pre>
<pre class="brush:java"><classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"><attributes><attribute name="maven.pomderived" value="true"><pre class="programlisting" style="background-color: #eeeeee; border: 1px dashed rgb(51, 51, 51); color: #000066; font-family: 'Courier New', Courier, mono; font-size: 12.222222328186px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 1em; width: 1151.70141601563px;"><strong class="userinput"><code style="font-family: 'Courier New', Courier, mono;">curl "http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip" -o "ec2-api-tools.zip"</code></strong></pre>
<div class="p1">
<span class="s1">obrienlabs-mbp15:aws michaelobrien$ ec2-describe-instances</span></div>
</attribute></attributes></classpathentry></pre>
<pre class="brush:java"><classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"><attributes><attribute name="maven.pomderived" value="true">
</attribute></attributes></classpathentry></pre>
<pre class="brush:java"><classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"><attributes><attribute name="maven.pomderived" value="true"><a href="http://docs.aws.amazon.com/cli/latest/userguide/installing.html">http://docs.aws.amazon.com/cli/latest/userguide/installing.html</a></attribute></attributes></classpathentry></pre>
<pre class="brush:java"><classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"><attributes><attribute name="maven.pomderived" value="true"><attribute name="org.eclipse.jst.component.dependency" value="/WEB-INF/lib"><pre class="programlisting" style="background-color: #eeeeee; border: 1px dashed rgb(51, 51, 51); color: #000066; font-family: 'Courier New', Courier, mono; font-size: 12.222222328186px; margin-bottom: 5px; margin-top: 5px; overflow: auto; padding: 1em; width: 1189.70483398438px;">curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"</pre>
</attribute></attribute></attributes>
<div class="p1">
<span class="s1">obrienlabs-mbp15:aws michaelobrien$ aws --version</span></div>
<div class="p1">
<span class="s1">aws-cli/1.7.0 Python/2.7.6 Darwin/14.0.0</span></div>
</classpathentry>
<div>
</div>
<div>
Install Apache HTTP Server</div>
<div>
<a href="http://httpd.apache.org/docs/trunk/install.html">http://httpd.apache.org/docs/trunk/install.html</a></div>
<div>
</div>
<div>
Install Apache Tomcat Web Container</div>
<div>
When registering a domain with a Canadian address - you will need a space in the postal code to get past the verification check on AWS Route53</div>
EC2
Get metadata using the<a href="http://en.wikipedia.org/wiki/Link-local_address#IPv4"> link local</a> address
<a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html">http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html</a>
</pre>
<pre class="brush:java">Increase the upload size for files greater than 5Mb in server.xml by changing the maxSwallowSize</pre>
<pre class="brush:java"></pre>
<pre class="brush:java"> <Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
maxSwallowSize="52428800"
redirectPort="8443" /></pre>
<pre class="brush:java">Install monitoring services like SumoLogic, New Relic and <a href="http://www.boundary.com/">Boundary</a></pre>
<pre class="brush:java"></pre>
<pre class="brush:java"><span style="background-color: #eaebec; color: #666666; font-size: 14px; line-height: 21px; white-space: normal;">curl -fsS -d '{"token":"api.6f44444444-3333"}' -H 'Content-Type: application/json' https://meter.boundary.com/setup_meter > setup_meter.sh && chmod +x setup_meter.sh && ./setup_meter.sh</span><h3>
</h3>
Issues</pre>
<pre class="brush:java">20150408: beanstalk environment configuration update does not work - it fails to update the AMI</pre>
<pre class="brush:java">before ami: ami-986327f0
after update from 2014.09 v1.2.0 to 2015.03 v1.3.0
after ami: ami-986327f0
2015-04-08 11:45:33 UTC-0400 ERROR Update environment operation is complete, but with errors. For more information, see troubleshooting documentation.
Since the default ami for 2015.03 v1.3.0 is
ami-0c6f5f64
This should be the ami during the upgrade
Fix: replace the ami with the new ami-0c6f5f64 version after the AWS upgrade
2015-04-08 15:13:12 UTC-0400 INFO Environment health has transitioned from RED to GREEN
<h3>
AWS Technical Links</h3>
Beanstalk CloudFormation parameters - http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options.html
<div>
Elastic Container Service - http://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html<br />
http://aws.amazon.com/blogs/aws/ec2-container-service-in-action/?sc_ichannel=ha&sc_ipage=homepage&sc_icountry=en&sc_isegment=c&sc_iplace=hero1&sc_icampaigntype=product_launch&sc_icampaign=ha_en_ECS_Launch&sc_icategory=none&sc_idetail=ha_en_281_1&sc_icontent=ha_281&<br />
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_GetStarted.html</div>
Send Mail - http://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-using-smtp-java.html
SES Sending Limits -
https://console.aws.amazon.com/ses/home?region=us-east-1#dashboard:
Installing AWS CLI - <a href="http://docs.aws.amazon.com/cli/latest/userguide/installing.html">http://docs.aws.amazon.com/cli/latest/userguide/installing.html</a></pre>
<pre class="brush:java"></pre>
<pre class="brush:java">configure secure access</pre>
<pre class="brush:java"><a href="http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_VirtualMFA.html">http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_VirtualMFA.html</a></pre>
<pre class="brush:java"></pre>
<pre class="brush:java">Get all the hosts up on your private subnet</pre>
<pre class="brush:java"><div class="p1">
<span class="s1"></span>
<br />
<pre><span class="s1">for ip in $(seq 1 254); do ping -c 1 192.168.0.$ip>/dev/null; [ $? -eq 0 ] && echo "192.168.0.$ip UP" || : ; done</span></pre>
<pre><span class="s1">
</span></pre>
<pre><span class="s1"><span style="font-family: 'Times New Roman'; white-space: normal;">Notes:</span></span></pre>
<pre><span class="s1"><span style="font-family: 'Times New Roman'; white-space: normal;">20150329: RDS Performance: My t2-micro RDS instances are taking 13 sec for all queries (count, select, filter) where they used to take less than 1. Rebooting - no change, rebuilding the RDS instance decreases the time to 9 sec (indexing?)</span></span></pre>
<pre><span class="s1"><span style="font-family: 'Times New Roman'; white-space: normal;">When I switched up to a t2-small with double the ram - speed increased to 6 sec. Therefore I have hit some sweet spot with my 3 million biometric records.
After a couple warm up queries (directly in MySQL Workbench) to kick in burst mode for "select count(1) from biometric.gps_record"
t2-micro = 11.9 sec
t2-small = 2.5 to 1.1 sec
t2-medium = 1.6 to 0.9 sec </span></span></pre>
<pre><span class="s1"><span style="font-family: 'Times New Roman'; white-space: normal;">I took a T2.medium (2 threads & 4G) and modified it back to a T2.micro (1 thread @ 1G) and performance went from 1.1 sec to 10 sec as expected - as the query is memory intensive and peaks at 1700Mb from a baseline of 450Mb - therefore a T2.small (1 thread @ 2G) will sufficed until I run up 6 million records</span></span></pre>
<pre><span class="s1"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></span></pre>
<pre><span class="s1"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></span></pre>
</div>
</pre>
<h2>
<br /></h2>
</div>
<div>
<div class="p1">
<span class="s1"><br /></span></div>
</div>
Michael O'Brienhttp://www.blogger.com/profile/14907623981077693781noreply@blogger.com1