Showing posts with label Java Performance Tuning. Show all posts
Showing posts with label Java Performance Tuning. Show all posts
30 May 2014
11 June 2013
How Riak handles Incremental Infinite Scalibility
http://docs.basho.com/riak/latest/tutorials/fast-track/What-is-Riak/
Riak Automatically Re-Distributes Data When Capacity is Added
When you add machines, data is rebalanced automatically with no downtime. New machines take responsibility for their share of data by assuming ownership of some of the partitions; existing cluster members hand off the relevant partitions and the associated data. The new node continues claiming partitions until data ownership is equal, updating a picture of which nodes own what as it goes. This picture of cluster state is shared to every node using a gossip protocol and serves as a guide to route requests. This is what makes it possible for any node in the cluster to receive requests - developers don't need to deal with the underlying complexity of what data is where.
Incremental Scaliblity
This is how amazon dynamo handles elastically incremental scalability
pdf version http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf
http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
pdf version http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf
http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
4.2 Partitioning Algorithm
One of the key design requirements for Dynamo is that it must scale incrementally. This requires a mechanism to dynamically partition the data over the set of nodes (i.e., storage hosts) in the system. Dynamo’s partitioning scheme relies on consistent hashing to distribute the load across multiple storage hosts. In consistent hashing [10], the output range of a hash function is treated as a fixed circular space or “ring” (i.e. the largest hash value wraps around to the smallest hash value). Each node in the system is assigned a random value within this space which represents its “position” on the ring. Each data item identified by a key is assigned to a node by hashing the data item’s key to yield its position on the ring, and then walking the ring clockwise to find the first node with a position larger than the item’s position. Thus, each node becomes responsible for the region in the ring between it and its predecessor node on the ring. The principle advantage of consistent hashing is that departure or arrival of a node only affects its immediate neighbors and other nodes remain unaffected.
The basic consistent hashing algorithm presents some challenges. First, the random position assignment of each node on the ring leads to non-uniform data and load distribution. Second, the basic algorithm is oblivious to the heterogeneity in the performance of nodes. To address these issues, Dynamo uses a variant of consistent hashing (similar to the one used in [10, 20]): instead of mapping a node to a single point in the circle, each node gets assigned to multiple points in the ring. To this end, Dynamo uses the concept of “virtual nodes”. A virtual node looks like a single node in the system, but each node can be responsible for more than one virtual node. Effectively, when a new node is added to the system, it is assigned multiple positions (henceforth, “tokens”) in the ring. The process of fine-tuning Dynamo’s partitioning scheme is discussed in Section 6.
Using virtual nodes has the following advantages:
- If a node becomes unavailable (due to failures or routine maintenance), the load handled by this node is evenly dispersed across the remaining available nodes.
- When a node becomes available again, or a new node is added to the system, the newly available node accepts a roughly equivalent amount of load from each of the other available nodes.
- The number of virtual nodes that a node is responsible can decided based on its capacity, accounting for heterogeneity in the physical infrastructure.
20 October 2012
04 October 2012
memory cpu monitoring
mpstat -p 2 2222222222222222
jstat -gcutil -h -t 20 17328 2000
for different options http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html
jstat -gcnew -t -h10 10s 0
-gcold
-gcnewcapacity
-gcoldcapacity
-gcpermcapacity
-gc
-gcutil
jstat -gcnew -t -h10
-gcold
-gcnewcapacity
-gcoldcapacity
-gcpermcapacity
-gc
-gcutil
11 September 2012
ENABLE JMX for remote cpu monitoring
private static final String JMX_ENABLED = "-Dcom.sun.management.jmxremote=true";
private static final String JMX_PORT = "-Dcom.sun.management.jmxremote.port=1200";
private static final String SSL ="-Dcom.sun.management.jmxremote.ssl=false";
private static final String AUTHENTICATE ="-Dcom.sun.management.jmxremote.authenticate=false";
09 September 2012
jvisualvm remote monitoring
In order to monitor your remote app with jvisualvm which comes with the jdk
(location /usr/java/jdk1.6.0_30/bin)
first you should run ./jstatd in the remote server and give port like the one below;
./jstatd -p 1111 -J-Djava.security.policy=visualvm.policy
visualvm.policy is a file contains the following lines
grant codebase "file:${java.home}/../lib/tools.jar" {
permission java.security.AllPermission;
};
security.policy is needed not to hit the following error
Could not create remote object
access denied (java.util.PropertyPermission java.rmi.server.ignoreSubClasses write)
java.security.AccessControlException: access denied (java.util.PropertyPermission java.rmi.server.ignoreSubClasses write)
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:374)
at java.security.AccessController.checkPermission(AccessController.java:546)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
at java.lang.System.setProperty(System.java:725)
at sun.tools.jstatd.Jstatd.main(Jstatd.java:122)
http://download.java.net/jdk8/docs/technotes/tools/share/jstatd.html
http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html#examples
http://java.sun.com/performance/jvmstat/
20 March 2009
ALL I LEARNED
Hi,
I am dealing with JEE and Java Development since 2006 June, in a large-scale marketing&telco company.
Time to market companies have common property, this is speed.
This speed word, can harm software development soul therefore it is quite hard to be a soldier in such an army. But this teaches you; discipline should be your guide.
My experiences, mistakes, faults, successes teach me several rules, in this very short(or very long) time period.
I've decided to note tens of them on my web log. If you want to earn yourlife as a developer, at least you should read them.
1. Never give a deadline , until you understand what is your task.
2. Never start to code , until you pretty sure about your development plan.
3. Arrange meeting(s) to yourself for just thinking on exceptions after development ends, and while development continues.
4. Test it, Do Test it, Do Test What you create. Test Yourself.
Never, Never, Never Trust Your QA Member.
5. Documantate what is going on, with all details but with 0(zero) unnecessary information. Your document should not be documented at the end. It should be a living period. It should start at the first day and you will not say documentation is finished until you put your code to the production.
6. Never give any support until He or She(Analyst,Tester,Developer,Support Member,Operation Contact..) right to ask it.
7. Never forget to report, any unplanned task.(Mail,Defect,Question)
8. Feel free to remove any detail,that makes you un-disciplined
9. Be Cool, Never hurt any of your colleague. Please do not forget, any of us can make mistake. Mistakes are temporary, but words are long lasting.
10. Lastly, but certainly not the least; Keep Yourself UPDATE!!! Always search for innovation.
I know the question will come as; WHEN?
Believe, it is more expensive otherwise :)
I am dealing with JEE and Java Development since 2006 June, in a large-scale marketing&telco company.
Time to market companies have common property, this is speed.
This speed word, can harm software development soul therefore it is quite hard to be a soldier in such an army. But this teaches you; discipline should be your guide.
My experiences, mistakes, faults, successes teach me several rules, in this very short(or very long) time period.
I've decided to note tens of them on my web log. If you want to earn yourlife as a developer, at least you should read them.
1. Never give a deadline , until you understand what is your task.
2. Never start to code , until you pretty sure about your development plan.
3. Arrange meeting(s) to yourself for just thinking on exceptions after development ends, and while development continues.
4. Test it, Do Test it, Do Test What you create. Test Yourself.
Never, Never, Never Trust Your QA Member.
5. Documantate what is going on, with all details but with 0(zero) unnecessary information. Your document should not be documented at the end. It should be a living period. It should start at the first day and you will not say documentation is finished until you put your code to the production.
6. Never give any support until He or She(Analyst,Tester,Developer,Support Member,Operation Contact..) right to ask it.
7. Never forget to report, any unplanned task.(Mail,Defect,Question)
8. Feel free to remove any detail,that makes you un-disciplined
9. Be Cool, Never hurt any of your colleague. Please do not forget, any of us can make mistake. Mistakes are temporary, but words are long lasting.
10. Lastly, but certainly not the least; Keep Yourself UPDATE!!! Always search for innovation.
I know the question will come as; WHEN?
Believe, it is more expensive otherwise :)
18 March 2007
TAKE CARE ON TYPE CONVERSION --- STRING TO DOUBLE AND NUMBER TO DOUBLE
I consider about the cost of type conversion in Java.
And I test it with a 1 million call.And Understand that
if you well choose your type for conversion then you gain
from performance. :)
My Considirarion point is about converting to double.
Javas popular class String has flexibity and this flexibity makes
many developers to use it wherever possible.
But My Experiment shows that: Take Care On Scaleble Stuff ;)
I mean if you have the Number Class or sub types of it,
it will have dramatic effect over using String.
See my Code.
Timer Class for measuring Time.
Converter Class.
And Test Class.
Results On Intel(R) Pentium(R) M processor 2,00 Ghz
798 Mhz 1 Gb Ram PC
First Run:
----------------------------------------
Total Time Elapsed for STRING is : 375
Total Time Elapsed for int is : 32
Total Time Elapsed for float is : 15
Total Time Elapsed for NUMBER is : 16
Second Run
----------------------------------------
Total Time Elapsed for STRING is : 406
Total Time Elapsed for int is : 16
Total Time Elapsed for float is : 15
Total Time Elapsed for NUMBER is : 16
Third Run
------------------------------------------
Total Time Elapsed for STRING is : 406
Total Time Elapsed for int is : 31
Total Time Elapsed for float is : 31
Total Time Elapsed for NUMBER is : 32
Fourth Run
--------------------------------------------
Total Time Elapsed for STRING is : 329
Total Time Elapsed for int is : 31
Total Time Elapsed for float is : 15
Total Time Elapsed for NUMBER is : 16
Fifth Run
--------------------------------------------
Total Time Elapsed for STRING is : 359
Total Time Elapsed for int is : 16
Total Time Elapsed for float is : 16
Total Time Elapsed for NUMBER is : 15
Quick Result : If possible, You should prefer using number type or subtypes of it rather Than using String for conversion
And I test it with a 1 million call.And Understand that
if you well choose your type for conversion then you gain
from performance. :)
My Considirarion point is about converting to double.
Javas popular class String has flexibity and this flexibity makes
many developers to use it wherever possible.
But My Experiment shows that: Take Care On Scaleble Stuff ;)
I mean if you have the Number Class or sub types of it,
it will have dramatic effect over using String.
See my Code.
Timer Class for measuring Time.
Converter Class.
And Test Class.
Results On Intel(R) Pentium(R) M processor 2,00 Ghz
798 Mhz 1 Gb Ram PC
First Run:
----------------------------------------
Total Time Elapsed for STRING is : 375
Total Time Elapsed for int is : 32
Total Time Elapsed for float is : 15
Total Time Elapsed for NUMBER is : 16
Second Run
----------------------------------------
Total Time Elapsed for STRING is : 406
Total Time Elapsed for int is : 16
Total Time Elapsed for float is : 15
Total Time Elapsed for NUMBER is : 16
Third Run
------------------------------------------
Total Time Elapsed for STRING is : 406
Total Time Elapsed for int is : 31
Total Time Elapsed for float is : 31
Total Time Elapsed for NUMBER is : 32
Fourth Run
--------------------------------------------
Total Time Elapsed for STRING is : 329
Total Time Elapsed for int is : 31
Total Time Elapsed for float is : 15
Total Time Elapsed for NUMBER is : 16
Fifth Run
--------------------------------------------
Total Time Elapsed for STRING is : 359
Total Time Elapsed for int is : 16
Total Time Elapsed for float is : 16
Total Time Elapsed for NUMBER is : 15
Quick Result : If possible, You should prefer using number type or subtypes of it rather Than using String for conversion
09 January 2007
StringBuffer versus String
This is typic performace consideration in Java
If code is slow, developer starts tuning with using StringBuffer
therefore i find suitable to post this issue..
String str = " alem buysa";
str += "kral fener";
This is typic innocent usage of '+' for concatanation.
if StringBuffer is int. to be used than
StringBuffer str = new StringBuffer("alem buysa");
str.append("kral fener");
if you look without thinking(like ox),then i am pretty sure you will say
first one is more effective since we use only + in the second case
we are calling a function
but developers look code with thinking deeply
lets evaluate byte code of it
this is for String class
0 new #7
3 dup
4 ldc #2
6 invokespecial #12
9 astore_1
10 new #8
13 dup
14 aload_1
15 invokestatic #23
18 invokespecial #13
21 ldc #1
23 invokevirtual #15
26 invokevirtual #22
29 astore_1
and for string buffer
0 new #8
3 dup
4 ldc #2
6 invokespecial #13
9 astore_1
10 aload_1
11 ldc #1
13 invokevirtual #15
16 pop
as you see in + operation type casting to String Buffer calling function and again returning to String class kills the performance
StringBuffer has significant performance gain compare to String..
(To be honest , usage of + is so easy..)
If code is slow, developer starts tuning with using StringBuffer
therefore i find suitable to post this issue..
String str = " alem buysa";
str += "kral fener";
This is typic innocent usage of '+' for concatanation.
if StringBuffer is int. to be used than
StringBuffer str = new StringBuffer("alem buysa");
str.append("kral fener");
if you look without thinking(like ox),then i am pretty sure you will say
first one is more effective since we use only + in the second case
we are calling a function
but developers look code with thinking deeply
lets evaluate byte code of it
this is for String class
0 new #7
3 dup
4 ldc #2
6 invokespecial #12
9 astore_1
10 new #8
13 dup
14 aload_1
15 invokestatic #23
18 invokespecial #13
21 ldc #1
23 invokevirtual #15
26 invokevirtual #22
29 astore_1
and for string buffer
0 new #8
3 dup
4 ldc #2
6 invokespecial #13
9 astore_1
10 aload_1
11 ldc #1
13 invokevirtual #15
16 pop
as you see in + operation type casting to String Buffer calling function and again returning to String class kills the performance
StringBuffer has significant performance gain compare to String..
(To be honest , usage of + is so easy..)
Multiplication and Division
Why we took Course Computer Organization?
Why we learned ALU?
Lets remember ;)
for (val = 0; val < 100000; val +=5) {
shiftX = val * 8;
myRaise = val * 2;
}
If we were to utilize bit shifting, performance would
increase up to six times. Here’s the refactored code:
for (val = 0; val < 100000; val += 5) {
shiftX = val << 3;
myRaise = val << 1;
}
Instead of multiplying by 8, we used the equivalent to shift
to the left (<<) by 3.
Each shift causes a multiplication by factors
of 2. The variable myRaise demonstrates this capability.
Shifting bits to the right (>>) is the same as dividing by factors
of 2. Of course this makes execution speed faster, but may
make it difficult for your peers to understand at a later date;
therefore it should be commented.
Reference : James McGrown
Why we learned ALU?
Lets remember ;)
for (val = 0; val < 100000; val +=5) {
shiftX = val * 8;
myRaise = val * 2;
}
If we were to utilize bit shifting, performance would
increase up to six times. Here’s the refactored code:
for (val = 0; val < 100000; val += 5) {
shiftX = val << 3;
myRaise = val << 1;
}
Instead of multiplying by 8, we used the equivalent to shift
to the left (<<) by 3.
Each shift causes a multiplication by factors
of 2. The variable myRaise demonstrates this capability.
Shifting bits to the right (>>) is the same as dividing by factors
of 2. Of course this makes execution speed faster, but may
make it difficult for your peers to understand at a later date;
therefore it should be commented.
Reference : James McGrown
Stop Being Clever
Many developers code with
reuse and flexibility in mind and
sometimes introduce additional
overhead into their programs.
At one time or another they’ve
written code similar to:
public void
doSomething(File file) {
FileInputStream fileIn = new FileInputStream(file);
// do something
}
It’s good to be flexible, but in this scenario they’ve created
more overhead. The idea behind doSomething is to manipulate
an InputStream, not a file, so it should be refactored as follows:
public void doSomething(InputStream inputStream){
// do something
}
reference:James McGovern
reuse and flexibility in mind and
sometimes introduce additional
overhead into their programs.
At one time or another they’ve
written code similar to:
public void
doSomething(File file) {
FileInputStream fileIn = new FileInputStream(file);
// do something
}
It’s good to be flexible, but in this scenario they’ve created
more overhead. The idea behind doSomething is to manipulate
an InputStream, not a file, so it should be refactored as follows:
public void doSomething(InputStream inputStream){
// do something
}
reference:James McGovern
Use Local Variables Whenever Possible
Arguments that are part of the method call and temporary
variables that are declared a part of this call are stored on the
stack, which is fast.
Variables such as static, instance, and new
objects are created on the heap, which is slower.
If you need high performance with the function you use
then do not try to call it from heap take it from the stack via localizing it..
And furthermore local variables are optimized also with the virtual machine that you are using
variables that are declared a part of this call are stored on the
stack, which is fast.
Variables such as static, instance, and new
objects are created on the heap, which is slower.
If you need high performance with the function you use
then do not try to call it from heap take it from the stack via localizing it..
And furthermore local variables are optimized also with the virtual machine that you are using
Make Classes Final Whenever Possible
Classes that are tagged as final can’t be extended. There are
many examples of this technique in the core Java APIs, such as
java.lang.String. Tagging the String class as final prevents
developers from creating their own implementation of the
length method.
Furthermore, if a class is final, all the methods of the class
are also final.
example usage: String class
many examples of this technique in the core Java APIs, such as
java.lang.String. Tagging the String class as final prevents
developers from creating their own implementation of the
length method.
Furthermore, if a class is final, all the methods of the class
are also final.
example usage: String class
Use Alternatives to the New Keyword
By creating an instance of a class
using the new keyword, all constructors in the chain are
called.
You may creating more than one object
Thik this case.. Result may be severe,
If you need to create a new instance of a class, you can
use the clone() method of an object that implements the
cloneable interface. The clone method doesn’t invoke any
class constructors.
If you’ve used design patterns as part of your architecture
and use the factory pattern to create objects, the change will
be simple. Listed below is the typical implementation of the
factory pattern.
public static Account getNewAccount() {
return new Account();
}
The refactored code using the clone method may look
something like this:
private static Account BaseAccount = new Account();
public static Account getNewAccount() {
return (Account) BaseAccount.clone();
}
The above thought process is also useful for the implementation
of arrays.
Reference:James McGovern
using the new keyword, all constructors in the chain are
called.
You may creating more than one object
Thik this case.. Result may be severe,
If you need to create a new instance of a class, you can
use the clone() method of an object that implements the
cloneable interface. The clone method doesn’t invoke any
class constructors.
If you’ve used design patterns as part of your architecture
and use the factory pattern to create objects, the change will
be simple. Listed below is the typical implementation of the
factory pattern.
public static Account getNewAccount() {
return new Account();
}
The refactored code using the clone method may look
something like this:
private static Account BaseAccount = new Account();
public static Account getNewAccount() {
return (Account) BaseAccount.clone();
}
The above thought process is also useful for the implementation
of arrays.
Reference:James McGovern
Don’t Initialize Variables Twice
Java by default initializes variables to a known value upon
calling the particular class’s constructor. All objects are set to
null, integers (byte, short, int, long) are set to 0, float and double
are set to 0.0, and Booleans are set to false.
This is especially
important if the class has been extended from another
class, as all chain constructors are automatically called when
creating an object with the new keyword.
Reference: James McGovern
calling the particular class’s constructor. All objects are set to
null, integers (byte, short, int, long) are set to 0, float and double
are set to 0.0, and Booleans are set to false.
This is especially
important if the class has been extended from another
class, as all chain constructors are automatically called when
creating an object with the new keyword.
Reference: James McGovern
Subscribe to:
Posts (Atom)