The Rise And Fall Of The Full Stack Developer | TechCrunch

SOURCE Techcrunch:



It seems as though everyone in tech today is infatuated with the full-stack developer. Full stack may have been possible in the Web 2.0 era, but a new generation of startups is emerging, pushing the limits of virtually all areas of software. From machine intelligence to predictive push computing to data analytics to mobile/wearable and more, it’s becoming virtually impossible for a single developer to program across the modern full stack.



When I first started programming computers as a kid in the pre-mobile, pre-web late 1970s/early 1980s, a single person typically wrote a complete software program from start to finish, and there weren’t many other layers of software between the programmer and the hardware. Using assembly language was the norm for programmers trying to squeeze more performance and space out of machines with 8-bit processors and very limited memory.



Programming applications quickly evolved into a team sport with the advent of client/server computing in the late 1980s and early 1990s, and the wave of Internet computing in the late 1990s and early 2000s. Each facet of new technology was so complex that a specialist was often required, sometimes one for different tiers (e.g. front-ends, databases, application servers, etc.) Managing a business website became a specialty that included operating networking equipment, such as routers and load balancers, tweaking Java virtual machines, and using various database indexing mechanisms.



By the mid-2000s, creating virtually anything — from simple websites to next-generation SaaS services — became prohibitively expensive. The rising expense was directly correlated to the overhead of numerous individuals from the various tiers communicating (and often miscommunicating) with each other, and changes in one tier cascading into other tiers and into deployment parameters. As Marc Andreessen pointed out in a recent tweet storm about burn rates, “More people multiplies communication overhead exponentially, slows everything down.”



Conversely, the technology to create the new generation of Web 2.0 sites became increasingly streamlined and simplified. Programmers switched from using the more complicated enterprise Java stack and databases such as Oracle to the more straightforward LAMP stack (Linux, Apache, MySQL, PHP/Python/Perl). New languages and frameworks such as Django and Ruby on Rails automated the layer between the website and the database. Front-end frameworks such as jQuery helped abstract all of the intricacies between different browsers. Cloud services such as Amazon Web Services simplified deployment and provided turnkey networking.



By the late 2000s, it became possible for many programmers to deliver a complete consumer or SaaS site, including a dynamic web client, server-side business logic, a scalable database, deployment, and operational support. This new breed of full-stack developer could run circles around teams of programmers attempting the same task. When projects scaled up, adding more full-stack programmers allowed a single person to add a single feature across all the tiers of an application, which accelerated feature delivery over the communication overhead of having different people own the feature in each tier.

If you’re building a website on the full stack illustrated above, find full-stack developers who can effectively wear these hats. But these days — and call me crazy — I’d consider this a less-than-full-stack. Here’s a fuller full stack:

I’d wager that there are zero individuals with advanced-level knowledge in each of these areas that would be capable of single-handedly delivering this next generation kind of application. Just keeping up with the advancements and new programming interfaces in each category is almost a full-time job.



We are in the midst of a rapid shift to more complicated technologies that, as in days gone by, require experts at each tier. Developing excellent iOS and Android applications requires experts in those platforms that understand the intricacies. Operationally, tending to new object databases such as Mongo requires constant attention and tweaking. Running an application on cloud services such as Amazon requires knowing the ins-and-outs of its various services and expertise on how to failover across regions. Even the venerable web front-end has evolved into CSS4, JSON and JavaScript MVC frameworks, such as Angular.js and Backbone.js.



In this brave new world, it is critical to have at least one person with at least a functional understanding of each of the composite parts who is also capable of connecting various tiers and working with each expert so that a feature can actually be delivered. In a way, these tier-connecting, bridge-building software architects — who are likely experts in only one or a couple of tiers — are less full stack developer and much more full stack integrator.



Rest in peace, full stack developers. Welcome, full stack integrators, in addition to engineers with deep technical skills in particular areas. It’s a fascinating world of software out there and we need you more than ever.

Mocks Aren't Stubs

Mocks Aren't Stubs:

The term 'Mock Objects' has become a popular one to describe special case objects that mimic real objects for testing. Most language environments now have frameworks that make it easy to create mock objects. What's often not realized, however, is that mock objects are but one form of special case test object, one that enables a different style of testing. In this article I'll explain how mock objects work, how they encourage testing based on behavior verification, and how the community around them uses them to develop a different style of testing.

Final Thoughts

As interest in unit testing, the xunit frameworks and Test Driven Development has grown, more and more people are running into mock objects. A lot of the time people learn a bit about the mock object frameworks, without fully understanding the mockist/classical divide that underpins them. Whichever side of that divide you lean on, I think it's useful to understand this difference in views. While you don't have to be a mockist to find the mock frameworks handy, it is useful to understand the thinking that guides many of the design decisions of the software.

The purpose of this article was, and is, to point out these differences and to lay out the trade-offs between them. There is more to mockist thinking than I've had time to go into, particularly its consequences on design style. I hope that in the next few years we'll see more written on this and that will deepen our understanding of the fascinating consequences of writing tests before the code.

Chris Wong's Development Blog: JmsTemplate is not evil

Chris Wong's Development Blog: JmsTemplate is not evil: A while back, I watched a presentation on JMS messaging where the presenter (Mark Richards) declared that Spring’s JmsTemplate is “evil,” an...

Java Spring bean with private constructor - Stack Overflow

Java Spring bean with private constructor - Stack Overflow:

Spring bean with private constructor

up vote4down votefavorite 1	Is possible in Spring that class for bean doesn't have public constructor but only private ? Will this private constructor invoked when bean is created? Thanks. java spring constructor javabeans private share|improve this question asked Aug 31 '11 at 8:36 user710818 2,47911340 77% accept rate
	3   How about giving it a try and then posting the results? – dm3 Aug 31 '11 at 8:38 It is difficult - I don't have all source. – user710818 Aug 31 '11 at 8:40 feedback

3 Answers

activeoldestvotes

up vote7down voteaccepted

Yes, Spring can invoke private constructors. If it finds a constructor with the right arguments, regardless of visibility, it will use reflection to set its constructor to be accessible. share|improve this answer edited Aug 31 '11 at 9:13 answered Aug 31 '11 at 8:48 Kevin Stembridge 2,271720

Could you please give me the reference to docs? Constructor has parameters, and in xml file to this arguments assigned values. – user710818 Aug 31 '11 at 8:52 you mean if a bean has specifically something like private bean() { ... } spring can invoke that? How is that possible, that defies the whole purpose of "private". – Ashkan Aryan Aug 31 '11 at 8:56 Sorry, I'm probably wrong about it only being no-arg constructors. I'm just going by what I've noticed on my own projects. I can't say I've ever seen it in the Spring documentation anywhere. But this is the javadoc for the class that is responsible for doing the instantiation. static.springsource.org/spring/docs/3.0.x/javadoc-api/org/…...) – Kevin Stembridge Aug 31 '11 at 9:01 4   @Ashkan It is possible using reflection. In particular, the Spring BeanUtils class uses a class called ReflectionUtils to make the constructor accessible. See javadocs for Constructor.setAccessible(). – Kevin Stembridge Aug 31 '11 at 9:03 @Kevin Stembridge you are correct, it is possible using Reflection. – Ashkan Aryan Aug 31 '11 at 9:07 show 2 more comments feedback

up vote2down vote

You can always use a factory method to create beans rather than relying on a default constructor, fromThe IoC container: Instantiation using an instance factory method: <!-- the factory bean, which contains a method called createInstance() --> <bean id="serviceLocator" class="com.foo.DefaultServiceLocator"> <!-- inject any dependencies required by this locator bean --> </bean> <!-- the bean to be created via the factory bean --> <bean id="exampleBean" factory-bean="serviceLocator" factory-method="createInstance"/> This has the advantage that you can use non-default constructors for your bean, and the dependencies for the factory method bean can be injected as well. share|improve this answer edited Aug 31 '11 at 9:08 answered Aug 31 '11 at 8:51 Matthew Farwell 20.9k52964

interesting, but I cannot find factory. May be exists another way? – user710818 Aug 31 '11 at 9:08 1   You can't find factory? I don't understand what you are saying. – Matthew Farwell Aug 31 '11 at 9:09 @Matthew: He appears to be retrofitting Spring around some fugly third-party code. That's just bound to be messy. – Donal Fellows Aug 31 '11 at 9:29 1   @Donal, no I didn't understand what he said in the comment. – Matthew Farwell Aug 31 '11 at 9:52

'via Blog this'

Java 7 Webcast

Java 7 Webcast:

JAVA 7 TECHNICAL BREAKOUTS				EVENT HOSTS
	See the presentations for all technical breakouts below: Making Heads and Tails of Project Coin, Small Language Changes in JDK 7 (Presentation/PDF) Divide and Conquer Parallelism with the Fork/Join Framework (Presentation/PDF) The New File System API in JDK 7 (Presentation/PDF) A Renaissance VM: One Platform, Many Languages (Presentation/PDF)

'via Blog this'

Work-Stealing & Recursive Partitioning with Fork/Join - igvita.com

Work-Stealing & Recursive Partitioning with Fork/Join

By Ilya Grigorik on February 29, 2012

Implementing an efficient parallel algorithm is, unfortunately, still a non-trivial task in most languages: we need to determine how to partition the problem, determine the optimal level of parallelism, and finally build an implementation with minimal synchronization. This last bit is especially critical since as Amdahl's law tells us: "the speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program".

The Fork/Join framework (JSR 166) in JDK7 implements a clever work-stealing technique for parallel execution that is worth learning about - even if you are not a JDK user. Optimized for parallelizing divide-and-conquer (and map-reduce) algorithms it abstracts all the CPU scheduling and work balancing behind a simple to use API.

Load Balancing vs. Synchronization

One of the key challenges in parallelizing any type of workload is the partitioning step: ideally we want to partition the work such that every piece will take the exact same amount of time. In reality, we often have to guess at what the partition should be, which means that some parts of the problem will take longer, either because of the inefficient partitioning scheme, or due to some other, unanticipated reasons (e.g. external service, slow disk access, etc).

This is where work-stealing comes in. If some of the CPU cores finish their jobs early, then we want them to help to finish the problem. However, now we have to be careful: trying to "steal" work from another worker will require synchronization, which will slowdown the processing. Hence, we want work-stealing, but with minimal synchronization - think back to Amdahl's law.

Fork/Join Work-Stealing

The Fork-Join framework (docs) solves this problem in a clever way: recursive job partitioning, and a double-ended queue (deque) structure for holding the tasks.

Given a problem, we divide the problem into N large pieces, and hand each piece to one of the workers (2 in the diagram above). Each worker then recursively subdivides the first problem at the head of the deque and appends the split tasks to the head of the same deque. After a few iterations we will end up with some number of smaller tasks at the front of the deque, and a few larger and yet to be partitioned tasks on end. So far so good, but what do we get?

Imagine the second worker has finished all of its work, while the first worker is busy. To minimize synchronization the second worker grabs a job from the end of the deque (hence the reason for efficient head and tail access). By doing so, it will get the largest available block of work, allowing it to minimize the number of times it has to interact with the other worker (aka, minimize synchronization). Simple, but a very clever technique!

Fork-Join in Practice (JRuby)

It is important to understand why and how the Fork/Join framework works under the hood, but the best part is that the API presented to the developer completely abstracts all of these details. The runtime can and will determine the level of parallelism, as well as handle all the work of balancing tasks across the available workers:

require 'forkjoin'

class Fibonacci < ForkJoin::Task
  def initialize(n)
    @n = n
  end

  def call
    return @n if @n <= 1

    (f = Fibonacci.new(@n - 1)).fork
    Fibonacci.new(@n - 2).call + f.join
  end
end

n = ARGV.shift.to_i
pool = ForkJoin::Pool.new # 2,4,8, ...

puts "fib(#{n}) = #{pool.invoke(Fibonacci.new(n))}, parallelism = #{pool.parallelism}"

# $> ruby fib.rb 33

The JRuby forkjoin gem is a simple wrapper for the Java API. In the example above, we instatiate aForkJoin::Pool and call invoke passing it our Fibonacci problem. The Fibonacci problem is type ofForkJoin::Task, which implements a recursive call method: if the problem is "too big", then we split it into two parts, one of which is "forked" (pushed onto the head of the deque), and the second half we invoke immediately. The final answer is the sum of the two tasks.

By default, the ForkJoin::Pool will allocate the same number of threads as available CPU cores - in my case, that happens to be 2, but the code above will automatically scale up to the number of available cores! Copy the code, run it, and you will see all of your available resources light up under load.

Map-Reduce and Fork-Join

The recursion of the divide-and-conquer technique is what enables the efficient deque work-stealing. However, it is interesting to note that the "map-reduce" workflow is simply a special case of this pattern. Instead of recursively partitioning the problem, the map-reduce algorithm will subdivide the problem upfront. This, in turn, means that in the case of an unbalanced workload we are likely to steal finer-grained tasks, which will also lead to more need for synchronization - if you can partition the problem recursively, do so!

require 'zlib'
require 'forkjoin'
require 'archive/tar/minitar'

pool = ForkJoin::Pool.new

jobs = Dir[ARGV[0].chomp('/') + '/*'].map do |dir|
  Proc.new do
    puts "Threads: #{pool.active_thread_count}, #{Thread.current} processing: #{dir}"

    backup = "/tmp/backup/#{File.basename(dir)}.tgz"
    tgz = Zlib::GzipWriter.new(File.open(backup, 'wb'))
    Archive::Tar::Minitar.pack(dir, tgz)

    File.size(backup)
  end
end

results = pool.invoke_all(jobs).map(&:get)
puts "Created #{results.size} backup archives, total bytes: #{results.reduce(:+)}"

# $> ruby backup.rb /home/igrigorik/

The above is a simple, map-reduce example via the same API. Given a path, this program will iterate over all files and folders, create all the backup tasks upfront, and finally invoke them to create the backup archives. Best of all, there is no threadpool management or synchronization code to be seen and the framework will easily pin all of your available cores, as well as automatically balance the work between all of the workers.

Parallel Java & Distributed Work-Stealing

The Fork/Join framework is deceivingly simple on the surface, but as usual, the devil is in the details when it comes to optimizing for performance. However, regardless of whether you are a JDK user or not, the deque, combined with a recursive partitioning step is a great pattern to keep in mind. The JDK implementation is built for "within single JVM" workloads, but a similar pattern can be just as useful in many distributed cases.

'via Blog this'