Showing posts with label Java. Show all posts
Showing posts with label Java. Show all posts

25 February 2015

Build Rocksdb Java API

sudo apt-get install git g++ automake openjdk-7-jdk curl
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
git clone https://github.com/facebook/rocksdb.git
cd rocksdb/
sudo apt-get install libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev
make rocksdbjava

This will produce 

/rocksdb/java/target/rocksdbjni-3.10.0-linux64.jar
/rocksdb/java/target/librocksdbjni-linux64.so

15 August 2014

My Top 10 Open Source People and Projects.

This is ordered in terms of how much I appreciate them..

I will not reveal if this list is in ascending or descending order :)

and my main evaluation criteria

a. is it working
b. is it useful
c. is it performing
d. is it documented
e. is it supported
f. is it simple



1. Curator (Jordan Zimmerman -- Netflix )

"guava is to Java what curator is to zookeeper."
I believe this statement just perfectly states what curator is..
Simple by providing examples, and you are confident to use by knowing that it is part of Netflix OSS  which is accounting for 34.2 downstream traffic in US.
For this project,  simplifies Zookeeper usage by wrapping it and providing an abstraction over it you even do not need reading documentation.
Although Jordan is the only person providing answers to the questions, you get them very quick whenever you raise smthg in the mail list.

2. Netty (Norman Maurer)

This simplifies network programming. very strong and very powerful in performance. Many(like storm, hornetq) open source projects already using it. It has a huge detailed documentation and Norman keep working on tuning it.


3. Zookeeper(Yahoo Engineering -- Apache 2 - Benjamin Reed& Flavio Junqueira)

This tool is a great problem solver and pain killer in distributed environment. Every problem you will meet is highly likely already met and solved. Gives you coordination ability.There is a big commiter list behind but I like these 2 guy because of the  book they published.


4. Esper (Thomas Bernhard - Gpl2)

I believe this is the number one CEP tool exists. Simple to use. very well documented.
And if it does not use any reflection Ill definitely put this one to the top of the list.
However performance wise it has a good bit room to go. May not be the perfect match for real time data analytic area if you target high(million per second event rate.) In that sense I would suggest you to write one according to exactly your needs. Implement your sliding window, pattern matching, filtering , enrichment algorithms in the way you like and just suiting your use case.
On the other hand if performance is not your number one requirement or you are ok to use more hardware, then use esper with its support license.


5. Hazelcast (Fuad Malikov)
I like to use publish subscribe model of the hazelcast. dead simple pain free tool developed by very skilled one.

6. Yammer - http://metrics.codahale.com/ - (Ryan Tenney )
Must tool for your instrumentation requirement.

7. logback (Ceki Gulcu)
I cannot imagine logging without slf4j, log4j and updated log4j => logback.
which means I cannot imagine logging without Ceki Gulcu's contributions.

8. Storm( Nathan Marz Backtype, Twitter )
if you are just starting to develop distributed real time application, you have a huge and long way to go.
But getting storm, and using it is dead simple. Even only you use it for prototype or testing reason this has a very good educational experience as well. Even you do not use storm just follow Nathans blog and twitter account :), he has a very strong mindset that can direct you better.
My favorite posts of him are
1. http://nathanmarz.com/blog/suffering-oriented-programming.html
2. http://nathanmarz.com/blog/the-mathematics-behind-hadoop-based-systems.html
3. http://nathanmarz.com/blog/interview-with-programmer-magazine.html

9. Exhibitor(Jordan Zimmerman - Netflix)

I d prefer to use this for monitoring zookeeper nodes and make sure they are running.
but you can do more.

Why I do like this project is when you are using Zookeeper , the main challenge is to explain what is it to the non technical people or semi technical people . However when you open exhibitor and show the zookeeper tree structured db, and telling them each jvm in your cluster is seeing this picture and this is only stored in a central replicated place then it will make sense, and doing this with a single command line is just priceless.

10. Guava (Joshua Bloch - Google Engineering )

Especially in software I do not like components or libraries that responsible more than one thing. But guava is like a big Util class which provides you all the helper functionality, controlling null, cache, collection api, string operations etc..
and Also knowing that Josuha Bloch, author of the Effective Java ,is influencing the feature set and implementation of it , makes  you extra confident to choose it in your dependency list.


btw I should note that all of these are Apache2 except Esper being GPLv2.

30 July 2014

Trade In Warrior

For the ones who confuses the log level severity always, I invent this :)

Trade In Warrior

TRACE << DEBUG << INFO << WARN << ERROR

30 May 2014

11 June 2013

How Riak handles Incremental Infinite Scalibility




http://docs.basho.com/riak/latest/tutorials/fast-track/What-is-Riak/





Riak Automatically Re-Distributes Data When Capacity is Added

When you add machines, data is rebalanced automatically with no downtime. New machines take responsibility for their share of data by assuming ownership of some of the partitions; existing cluster members hand off the relevant partitions and the associated data. The new node continues claiming partitions until data ownership is equal, updating a picture of which nodes own what as it goes. This picture of cluster state is shared to every node using a gossip protocol and serves as a guide to route requests. This is what makes it possible for any node in the cluster to receive requests - developers don't need to deal with the underlying complexity of what data is where.

Incremental Scaliblity

This is how amazon dynamo handles elastically incremental scalability

pdf version http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf


http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

4.2 Partitioning Algorithm

One of the key design requirements for Dynamo is that it must scale incrementally. This requires a mechanism to dynamically partition the data over the set of nodes (i.e., storage hosts) in the system. Dynamo’s partitioning scheme relies on consistent hashing to distribute the load across multiple storage hosts. In consistent hashing [10], the output range of a hash function is treated as a fixed circular space or “ring” (i.e. the largest hash value wraps around to the smallest hash value). Each node in the system is assigned a random value within this space which represents its “position” on the ring. Each data item identified by a key is assigned to a node by hashing the data item’s key to yield its position on the ring, and then walking the ring clockwise to find the first node with a position larger than the item’s position. Thus, each node becomes responsible for the region in the ring between it and its predecessor node on the ring. The principle advantage of consistent hashing is that departure or arrival of a node only affects its immediate neighbors and other nodes remain unaffected.
The basic consistent hashing algorithm presents some challenges. First, the random position assignment of each node on the ring leads to non-uniform data and load distribution. Second, the basic algorithm is oblivious to the heterogeneity in the performance of nodes. To address these issues, Dynamo uses a variant of consistent hashing (similar to the one used in [10, 20]): instead of mapping a node to a single point in the circle, each node gets assigned to multiple points in the ring. To this end, Dynamo uses the concept of “virtual nodes”. A virtual node looks like a single node in the system, but each node can be responsible for more than one virtual node. Effectively, when a new node is added to the system, it is assigned multiple positions (henceforth, “tokens”) in the ring. The process of fine-tuning Dynamo’s partitioning scheme is discussed in Section 6.
Using virtual nodes has the following advantages:
  • If a node becomes unavailable (due to failures or routine maintenance), the load handled by this node is evenly dispersed across the remaining available nodes.
  • When a node becomes available again, or a new node is added to the system, the newly available node accepts a roughly equivalent amount of load from each of the other available nodes.
  • The number of virtual nodes that a node is responsible can decided based on its capacity, accounting for heterogeneity in the physical infrastructure.



06 March 2013

javax.persistence.RollbackException: Error while committing the transaction


the valid jpa update entity code gives the exception below in the case of  wrong dependency( org.hibernate,  hibernate-core,  4.2.0.CR1)
   


        entityManager.getTransaction().begin();
        CompactDisc compactDisc = entityManager.find(CompactDisc.class, 7777);
        compactDisc.setTitle("ZZZZZZZ");
        entityManager.getTransaction().commit();
        entityManager.close();



Exception in thread "main" javax.persistence.RollbackException: Error while committing the transaction
    at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:92)
    at com.conygre.training.entities.LifecycleTester.main(LifecycleTester.java:25)
Caused by: java.lang.NullPointerException
    at org.hibernate.ejb.event.EJB3PostUpdateEventListener.handlePostUpdate(EJB3PostUpdateEventListener.java:71)
    at org.hibernate.ejb.event.EJB3PostUpdateEventListener.onPostUpdate(EJB3PostUpdateEventListener.java:67)
    at org.hibernate.action.internal.EntityUpdateAction.postUpdate(EntityUpdateAction.java:248)
    at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:205)
    at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:362)
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:354)
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:276)
    at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:328)
    at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:52)
    at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1212)
    at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:400)
    at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
    at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175)
    at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:75)


As a resolution Change the version to to 4.1.9.Final or 3.6.10.Final

04 October 2012

memory cpu monitoring


mpstat -p 2 2222222222222222 
jstat -gcutil -h -t 20 17328 2000 
for different options http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html 

jstat -gcnew -t -h10 10s 0
        -gcold
        -gcnewcapacity
        -gcoldcapacity
        -gcpermcapacity
        -gc
        -gcutil
        

11 September 2012

ENABLE JMX for remote cpu monitoring


 private static final String JMX_ENABLED = "-Dcom.sun.management.jmxremote=true";
    private static final String JMX_PORT = "-Dcom.sun.management.jmxremote.port=1200";
    private static final String SSL  ="-Dcom.sun.management.jmxremote.ssl=false";
    private static final String AUTHENTICATE  ="-Dcom.sun.management.jmxremote.authenticate=false";

09 September 2012

jvisualvm remote monitoring


In order to monitor your remote app with jvisualvm which comes with the jdk
(location /usr/java/jdk1.6.0_30/bin)

first you should run ./jstatd in the remote server and give port like the one below;

./jstatd -p 1111 -J-Djava.security.policy=visualvm.policy

visualvm.policy is a file contains the following lines

grant codebase "file:${java.home}/../lib/tools.jar" {
     permission java.security.AllPermission;
};


security.policy is needed not to hit the following error


Could not create remote object
access denied (java.util.PropertyPermission java.rmi.server.ignoreSubClasses write)
java.security.AccessControlException: access denied (java.util.PropertyPermission java.rmi.server.ignoreSubClasses write)
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:374)
        at java.security.AccessController.checkPermission(AccessController.java:546)
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
        at java.lang.System.setProperty(System.java:725)
        at sun.tools.jstatd.Jstatd.main(Jstatd.java:122)



http://download.java.net/jdk8/docs/technotes/tools/share/jstatd.html
http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html#examples
http://java.sun.com/performance/jvmstat/

08 June 2012

jvisualvm.exe

use jvisualvm under C:\Program Files\Java\jdk1.6.0_30\bin

18 October 2011

JBOSS is now OSGI Container

with jboss 7 released 2 months ago, osgi bundles can be deployed on it.


http://www.jboss.org/jbossas/osgi


The JBoss OSGi project provides the OSGi functionality in the JBoss Application Server.

The JBoss Application Server is now compliant with core OSGi specifications. This means that you can deploy OSGi Bundles in AS alongside your other AS deployments.

17 May 2011

no comment available

http://bellard.org/jslinux/

17 August 2010

source release 1.6 requires target release 1.6

If you are an IntelliJ IDEA user and you are using Maven in your project, and if you switch between 2 projects and theese projects are target to different versioned jres, then somehow intelliJ gives the error :"source release 1.6 requires target release 1.6"

if this is the case, working solition is this;

Open; File > Settings > Compiler > Java Compiler

Delete; text inside textfield labeled with "Additional Command Line Parameters"
(-target 1.5)

Then make intelliJ re-import your project.

09 July 2010

IDEA-25918 Error: Read from storage 0 bytes, but requested 12 bytes

While using intelliJ if your process closed suddenly, like killing process or power outage, you can see an error report below , on the console

Read from storage 0 bytes, but requested 12 bytes

This means you corrupt the caches,

You can solve this with File>invalidate Cache

if not work , go to the intelliJ compiler cache folder and delete caches there manually

Users\user\.IntelliJIdea90\system\caches

Users\user\.IntelliJIdea90\system\compiler\project\.caches

07 July 2010

Hibernate , Paging The Result

nice remedy to common pain , from hibernate :)

A commonly used technique is pagination. Users may see the result of their search
request (for example, for specific Items) as a page. This page shows a limited subset
(say, 10 Items) at a time, and users can navigate to the next and previous pages
manually. In Hibernate, Query and Criteria interfaces support this pagination of
the query result:

Query query =
session.createQuery("from User u order by u.name asc");
query.setMaxResults(10)
;

The call to setMaxResults(10) limits the query resultset to the first 10 objects
(rows) returned by the database. In this Criteria query, the requested page starts
in the middle of the resultset:

Criteria crit = session.createCriteria(User.class);
crit.addOrder( Order.asc("name") );
crit.setFirstResult(40);
crit.setMaxResults(20)
;

Querying with HQL and JPA QL

Starting from the fortieth object, you retrieve the next 20 objects. Note that
there is no standard way to express pagination in SQL—Hibernate knows the
tricks to make this work efficiently on your particular database. You can even add
this flexible pagination option to an SQL query. Hibernate will rewrite your SQL
for pagination:

Query sqlQuery =
session.createSQLQuery("select {u.*} from USERS {u}")
.addEntity("u", User.class);
sqlQuery.setFirstResult(40);
sqlQuery.setMaxResults(20);




From the bible : JavaPersistanceWithHibernate

28 June 2010

additivity="false" , log4j,logback,slf4j

On your log4j.xml or logback.xml , you can at most put 2 property near your logger definitions iff it is not root logger.(except level property) (Root logger does not take name , and additivity parameters.)

name is the name of logger, where you take when you initialize factory.

private static Logger loggerName = LoggerFactory.getLogger("XX.YY");

second property is additivity and i will mention about its importance.

this is very important.

When you are logging , your logs goes into two appenders ,

one: the specified appender
second: the root appender.

if this is the case you want , leave it out, since default additivity value is true and this makes your log write both root and specified logger.

But if you you do not want it to write root appender, (you do not want your log to write twice.)

Then place additivity false yo tour logger.

Your log will only write the specified logger. And you will gain 3-8 ms. for each.

Be careful on spelling.

True format is;

logger name="loggerName" additivity="false"



Do not write aditivity , :) it is additivity.

(I am insisting on it, since official logback site also has this defect. :))


Configuring Loggers, or the element

A logger is configured using the logger element. A logger element takes exactly one mandatory name attribute, an optional level attribute, and an optional aditivity attribute, which admits the values true or false. The value of the level attribute can be one of the case-insensitive strings TRACE, DEBUG, INFO, WARN, ERROR, ALL or OFF. The special case-insensitive value INHERITED, or its synonym NULL, will force the level of the logger to be inherited from higher up in the hierarchy. This comes in handy in case you set the level of a logger and later decide that it should inherit its level.

The logger element may contain zero or more appender-ref elements; each appender thus referenced is added to the named logger. It is important to keep mind that each named logger that is declared with a

28 May 2010

Logback with slf4j

______________________________________________
From: Osman
Sent: Friday, May 28, 2010 2:21 PM
To:
Subject:Logback with slf4j


Hocam,

özellikle * koyduguğum özelliklerine hayran kaldım. yazarı Ceki Gülcü, slf4j ile birlikte logback kullanmayı tavsiye ediyor.




Faster implementation
Based on our previous work on log4j, logback internals have been re-written to perform about ten times faster on certain critical execution paths. Not only are logback components faster, they have a smaller memory footprint as 

logback-classic speaks SLF4J natively

Automatic reloading of configuration files

Filters*
With logback, you have the option of keeping logging at the WARN level for all users except for the one user, say Alice, who is responsible for identifying the problem.

SiftingAppender
SiftingAppender can separate logging events according to user sessions, so that the logs generated by each user go into distinct log files, one log file per user.

Automatic compression of archived log files
RollingFileAppender can automatically compress archived log files during rollover. Compression always occurs asynchronously so that even for large log files, your application is not blocked for the duration of the compressio

Automatic removal of old log archives

Conditional processing of configuration files*


Developers often need to juggle between several logback configuration files targeting different environments such as development, testing and production. These configuration files have substantial parts in common differing only in a few places. To avoid duplication, logback supports conditional processing of configuration files with the help of ,  and  elements so that a single configuration file can adequately target several environments.


For further info :) : http://logback.qos.ch/index.html

03 January 2010

Invoke JDK Commands at runtime,

Now , To be honest for sometime i am writing a tool which is responsible for writing code, generating .dml files and creating a jar file , which is composed of them :))

if there are some lazy developers like me, i think this will make them happy :)


String[] options = new String[] {
"-classpath", "/", "-d", "\\mdb", "\\turkcell\\beawllog\\Exercise.java"
};


com.sun.tools.javac.Main.compile(options);

This line calls javac from tools.jar and generate .class file of the .java file..

Invoking other jdk commands as same as above..

Creating jar from java code is also pretty simple,..
Here is the statement which generates jar file MernisMDB.jar with combination of folders mdb META-INF,


String command = "jar cf MernisMDB.jar \\mdb \\META-INF ";
Runtime.getRuntime().exec(command);

..

if you have standart stuff you can generate class files and copy them into mdb folder
and you can copy your ejb-jar xml, weblogic-jar xml , beans.xml under META-INF folder


and package them under your jar file...


just easy

02 January 2010

OOP: Behavioural and Structural Constraints

Mark Needham answers & writes very well about the question, on What to Test?

OOP: Behavioural and Structural Constraints

He simple finds the definitions of functional and Behavioural constraints on the book
Advanced Object-Oriented Analysis and Design Using UML


Behavioural constraints limit the way object state changes may occur

Structural constraints limit the way objects associate with each other, that is, they restrict object state.

Two of his articles worth reading, and thinking on it..

http://www.markhneedham.com/blog/2009/09/02/tdd-test-the-behaviour-rather-than-implementation/
http://java.dzone.com/articles/oop-behavioural-and-structural?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+javalobby/frontpage+(Javalobby+/+Java+Zone)