A little summarizing Performance Note, JDK 7 Fork/join/ Parallelism, Performance/ Scaling - Horizontal/vErtical / Liveness issues

Before I start on the actual article I got side-tracked by what I was reading about Thrashing and where it occurs

Thrashing (computer science) - Wikipedia, the free encyclopedia: (in Main Memory)
 "Thrashing lowers the CPU utilization and sometime tends to zero."

Cache Thrashing
Thrashing in Network communication -  Silly Window problem in network communication - size of the payload becomes so small that the amount of overhead spend in processing the payload header has significantly increased (over 100% of) per payload hence the time spent in doing useful thing such as processing payload has become very inefficient.

And the N/w IO unit becomes busy doing it resulting in the computer doing less useful work against.

You might also like to see the following links with respect to Thrashing:


http://en.wikipedia.org/wiki/Working_set

http://en.wikipedia.org/wiki/Translation_lookaside_buffer

To attack the issue of Thrashing the best long term solution is to add more memory and
otherwise do the following things which will result in more/sufficient memory for a process in
execution so it doesn't thrash :

• Increase the amount of RAM in the computer (generally the best long-term solution).
• Decrease the number of programs being run on the computer.
• Replace programs that are memory-heavy with equivalents that use less memory.
• Improve spatial locality by replacing loops like:

  int m[256][256];
   
  for(i=0; i<=255; i++){
    for(k=0; k<=255; k++){
      m[k][i] = something();
    }
  }
  

  with

  int m[256][256];
   
  for(i=0; i<=255; i++){
    for(k=0; k<=255; k++){
      m[i][k] = something();
    }
  }

Herewith, I'd like to mention a small note (I have a bigger article hidden somewhere in my computer which I will try to pull it someday and add to this blog) which relates to the actual title of the article :-


Real Life comparison:
Real life example -  in case of  "good" software developers (who can build scale-able software) who are used to do production/support more than having them develop better software and scale the productivity of the organization. It is always better to use resources "efficiently (do more with less)" and the best way to do
that is do it at "max efficiency"; in other words deploy your good resources in places where their
"real talent" is used "at max efficiency". This talent/skill utilization scheme, when perfected can result
in an organization resulting in Max efficiency (an subsequently max capacity) at current costs.

Once you have max performance - then you should go for scaling (horizontally).

Don't lock your resources into doing "non-critical" stuff. But always do lock your resources to do
"critical stuff" else if a resource starts performing multiple critical activities - it won't be efficient
as it will result in heavy context switch (of his mental environment) and will eventually result in
sub-grade performance (This analogy is comparable to multi-threaded concurrent programs and
how well they are written to utilize the resources available to them) . One unseen resource or rather
intangible resource is "time" which is often either taken for granted or mis-understood for performance
or forgotten blissfully.

Increasing contention results in low CPU utilization and heavy context switching. And the liveness
(livelock/deadlock/starvation) problems that come along with this result in the CPU doing the less
of what it should be doing more for more efficiency. In such a case where the programs are not
well-concurrent in nature they are not easy to scale either. Because adding more resources (such as CPU/Memory) to increase the performance won't help since you are not solving the "contention" problem.

Before scaling - attack liveness / memory leak issue so that your system is performing efficiently (at max efficiency) with current resources. (take into consideration good divide and rule techniques anywhere and
everywhere you could deploy in the application /cache (distributed)/memory (distributed)/state partitioning /
partitioning in database/ transaction isolation levels (more granularity = more concurrency).
In other words, Design for performance when writing a new system (define your performance goals).
If improving the performance then solve the above problems to get whatever you can get  (remove liveness problems) partition data etc.  Scaling (adding resources should be the last measure) for improving performance. And always scale horizontally (its cheap and flexible) . You can easily scale up or scale down.
if you have varying load on your system (or load is seasonal). Vertical scaling has a huge price and you are
locked in and your application design is not leveraged here. Only hardware is doing the work of giving you
more performance so it also becomes costly. It is not easy to scale down. Also not flexible.

JDK 7 supplements the java.util.concurrent package with api for FORK/JOIN to take further leverage
concurrency capabilities in the your java  application by providing the capabilities of work-stealing
algorithms ( allows capability to break down your heavy long running tasks (recursively) to the a lowest common factor which the thread pool would work on - join the results recursively bottom-up and return the
result). This allows more concurrency because it allows the framework to break-down the tasks further.

In a non-work stealing multi-threaded environment some threads in your threadpool threads may sit idle
if there are no more new tasks to be run and the currently executing tasks by the busy threads in the Threadpool are substantially long-running. And if there is CPU available for these idle threads then
there is more scope for parallelism if you could break the long running tasks in the first place.

Fork/Join's purpose is just that. It helps you do more (in parallel) by allowing you to break down your tasks
so that the idle threads when done can "steal" (relative to the non fork/join multi-threaded world) the work.













'via Blog this'