oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: EMR cluster and selection of EC2 instance type - Cost Optimization!
http://oakgreen.blogspot.com/2015/06/emr-cluster-and-selection-of-ec2.html
Oak Green. Java. BigData. Wednesday, June 17, 2015. EMR cluster and selection of EC2 instance type - Cost Optimization! AWS Elastic MapReduce (EMR) is Amazon’s service providing Hadoop in the Cloud. EMR inherently uses the EC2 nodes as the hadoop nodes. While triggering an EMR cluster, we can choose appropriate instance type based on the requirements of resources and profile of hadoop process. In our case, we achieved direct cost saving of 30%. However this requires more effort in terms of ensuring fault...
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Collections in CQL3 - How they are stored
http://oakgreen.blogspot.com/2014/09/collections-in-cql3-how-they-are-stored_26.html
Oak Green. Java. BigData. Friday, September 26, 2014. Collections in CQL3 - How they are stored. If you don’t already know about collections in Cassandra CQL, following page provides excellent details about the same –. Http:/ www.datastax.com/dev/blog/cql3 collections. I have a big hangover of my extensive work with thrift API and hence I always get tempted to think how my CQL data looks like in internal storage structure. Users ( user id text. User id) ) cqlsh:dummy. Users; user id. 534eaca1452c11e49325...
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Subclipse plugin installation in Eclipse Indigo on RHEL- Bug and resolution
http://oakgreen.blogspot.com/2011/10/subclipse-plugin-installation-in.html
Oak Green. Java. BigData. Wednesday, October 5, 2011. Subclipse plugin installation in Eclipse Indigo on RHEL- Bug and resolution. I was facing issue while using subclipse plugin in my Eclipse Indigo on my RHEL (Redhat Linux) box. Installation was successful but was giving error while connecting with SVN. So the solution to the issue is providing JavaHL in eclipse classpath. Http:/ www.collab.net/downloads/subversion/redhat.html. CollabNet client contains required library (library name is. Java - getClas...
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Impact of NULL values on where-clause/group-by-clause in Hive queries
http://oakgreen.blogspot.com/2015/04/impact-of-null-values-on-where.html
Oak Green. Java. BigData. Wednesday, April 29, 2015. Impact of NULL values on where-clause/group-by-clause in Hive queries. Following is the check to verified that NULL values do not impact GROUP BY but it DOES IMPACT. Select count(*) from table1 where (field1 is NULL) AND dth = '2014-12-01-00' AND dth = '2014-12-01-23';. Select count(*) from table1 where (field1! Value1') AND dth = '2014-12-01-00' AND dth = '2014-12-01-23';. Subscribe to: Post Comments (Atom). IScribble - Sarang Anajwala. Silicon Valley...
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: How to Stop compaction for specific keyspace in cassandra
http://oakgreen.blogspot.com/2014/09/how-to-stop-compaction-for-specific.html
Oak Green. Java. BigData. Saturday, September 13, 2014. How to Stop compaction for specific keyspace in cassandra. How to stop compaction for a specific keyspace in cassandra? Reason behind the problem statement. Nodetool setcompactionthreshold keyspace cfname minthreshold maxthreshold. I am yet to try this). Apparently setting the min/max threshold value to 0 stops the compaction. Also, this is only possible from JMX and not from CLI! Yet to verify this. Subscribe to: Post Comments (Atom). Java - getCla...
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Eclipse Proxy settings - bug and workaround
http://oakgreen.blogspot.com/2011/10/eclipse-proxy-settings-bug-and.html
Oak Green. Java. BigData. Wednesday, October 5, 2011. Eclipse Proxy settings - bug and workaround. Many a times I experienced weird behavior of eclipse network connectivity (despite providing proxy settings) especially while installing plugins from update sites. However, didn’t pay enough heed to the issues and many a times moved on with installing the plugins by manually downloading it. As always, found the bug and work around easily by searching on net (community power! October 25, 2012 at 2:39 PM.
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Thread Count supported by JVM on various OSs
http://oakgreen.blogspot.com/2011/08/thread-count-supported-by-jvm-on.html
Oak Green. Java. BigData. Friday, August 19, 2011. Thread Count supported by JVM on various OSs. On increasing our JBoss heap-size to 2GB from 1GB, JBoss was crashing every 30-60 mins. On further analysis, it was found that the issue here was with number of threads. Second solution is to go for thread-pooling. Third solution, you can tune JVM thread stack space. Of course, ideally, all the 3 solutions should be used. However, third one is the quickest one to start with. Tuning JVM thread stack space:.
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Security - Array is stored directly
http://oakgreen.blogspot.com/2012/06/security-array-is-stored-directly.html
Oak Green. Java. BigData. Monday, June 25, 2012. Security - Array is stored directly. Sonar Violation: Security - Array is stored directly. Means: Constructors and methods receiving arrays should clone objects and store the copy. This prevents future changes from the user (caller/client) affect the internal functionality. Public void setMyArray(String[] myArray) {. ThismyArray = myArray;. Public void setMyArray(String[] newMyArray) {. If(newMyArray = null) {. ThismyArray = new String[0];. Silicon Valley ...
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Hive Query to get 95th Percentiled ranked item
http://oakgreen.blogspot.com/2014/10/hive-query-to-get-95th-percentiled.html
Oak Green. Java. BigData. Thursday, October 2, 2014. Hive Query to get 95th Percentiled ranked item. I was working on a query where I had to convert a complex MySql query which was providing 95. Percentile value from a group concat result. Following is the hive query to do the same on a simple sample table. Hive describe coll;. Col name data type. Hive select * from coll;. How percent rank() works? Status proc id proc rank desc. Started 4 0.0. Started 3 0.3333333333333333. Started 2 0.6666666666666666.
oakgreen.blogspot.com
Oak.... Green.... Java.... BigData: Compaction Strategy in cassandra
http://oakgreen.blogspot.com/2014/09/compaction-strategy-in-cassandra.html
Oak Green. Java. BigData. Friday, September 12, 2014. Compaction Strategy in cassandra. Cassandra supports two basic compaction strategies:. Before talking about these two compaction strategies, let’s take a look at what is compaction. In Cassandra, each write is written into MemTables and commitlogs in realtime and when Memtable fills up (or when manually memtables are flushed) data is written into the persistent files called SSTables. To avoid this issue with read operation, cassandra performs ‘c...