Wednesday, January 15, 2014

Sales are Good for You

I have just published the below presentation on Slideshare: half it's serious content, half it's just for fun.

Some of my best friends are present or former colleagues, most are developers but many of them are sales or marketing people. I was thinking about describing the often difficult relationship between software developers and sales people since many years now. Maybe I should give a talk about this, one of these days.

I am lucky I had the opportunity to work with very smart people, both on sales or IT, with the exception of some occasional moron, of course.

Hopefully you can recognize some common, funny patterns here.

Saturday, January 4, 2014

Book Review: "Real-time Web Application Development using Vert.x 2.0"

About Vert.x

I am following the evolution of the Vert.x project since its inception and I was expecting to see some books coming about the subject sooner or later. First of all, some explanation on what's Vert.x:
"Vert.x is a lightweight, high performance application platform for the JVM that's designed for modern mobile, web, and enterprise applications."
The product is 100% open source, licensed under the business friendly Apache Software License 2.0 and well documented. The project's founder and main maintainer is Tim Fox, of HornetQ fame.
Vert.x characteristics are nicely summarized in the related Wikipedia entry:
Vert.x is a polyglot event-driven application framework that runs on the Java Virtual Machine.
Similar environments written in other programming languages include Node.js for JavaScriptTwisted for PythonPerl Object Environment for Perllibevent for C and EventMachine for Ruby.
Vert.x exposes the API currently in JavaJavaScriptGroovyRuby and Python.
Scala and Clojure support is on the roadmap.
The application framework includes these features:
  • Polyglot. Application components can be written in Java, JavaScript, Groovy, Ruby or Python.
  • Simple concurrency model. All code is single threaded, freeing from the hassle of multi-threaded programming.
  • Simple, asynchronous programming model for writing truly scalable non-blocking applications.
  • Distributed event bus that spans the client and server side. The event bus even penetrates into in-browser JavaScript allowing to create so-called real-time web applications.
  • Module system and public module repository, to re-use and share components.

Few days ago I decided to purchase a recent book from Packt Publishing about the subject.

The Book

Language : English
Paperback : 122 pages
Release Date : September 2013
ISBN : 1782167951
ISBN 13 : 9781782167952
Author(s) : Tero Parviainen

I bought the eBook format only and I read it in both my iPad and computer screen, especially because the narration is mainly about building a fairly complete Web application, so I wanted to read and experiment at the same time, by entering the code by hand and running it step by step.

Below I provide a short summary of the book' sections and I will write some final comments about it.


The preface introduces clearly the content and objectives of the book
"Real-Time Web Application Development using Vert.x 2.0 will show you how to build a real-time web application with Vert.x. During the course of the book, you will go from the very first "Hello, World" to publishing a fully featured application on the Internet."
The scope is to build an editor with which people can create mind maps collaboratively. So, it is neither a too trivial application, nor something too complex. A good, balanced choice, with the additional bonus of learning a few of jQueryD3 JavaScript library and MongoDB on the road.

What the book covers 

Chapter 1, Getting Started with Vert.x, guides you through the installation of the Vert.x 2.0 platform and its prerequisites. In this chapter, you'll also write your very first Vert.x application: the web equivalent of "Hello, World".

Chapter 2, Developing a Vert.x Web Application, covers the development of a full-fledged Vert.x web application, including both the server and browser components. You will become familiar with the architecture of a typical Vert.x application.

Chapter 3, Integrating with a Database, extends the web application from the previous chapter by adding support for persisting data in a MongoDB database, using one of the available open source Vert.x modules: the MongoDB Persistor.

Chapter 4, Real-time Communication, builds on everything you've learned so far
to deliver the secret sauce: real-time communication. You will develop a real-time, collaborative, browser-based mind map editor.

Chapter 5, Polyglot Development and Modules, presents some of the polyglot features of Vert.x, as well as the development of reusable and distributable modules,
by creating a Java module that is used to save mind maps as PNG images.

Chapter 6, Deploying and Scaling Vert.x, shows how to deploy your Vert.x application on Internet, by setting up a Linux server with continuous deployment. Finally, we discuss the basics of scaling Vert.x for growing amounts of users and data. 

My comments

One might argue that the official Vert.x documentation is already quite comprehensive and this book doesn't add much, but conversely I feel this book allows for a smoother approach to learning, without the need to browse and search too much around the Web site. Maybe somebody with already a working experience with Vert.x wouldn't gain a lot of new knowledge here, at the end I feel the book is more focused on Vert.x beginners, but anyway it contains a good amount of advanced tips to make it valuable for a wide audience.

Vert.x is a polyglot framework, so one can choose among a set of popular programming languages, with even more on the move, but the code in the book is mainly Javascript, also server side. On one hand this seems to restrict a bit the potential audience, as to fully understand the source code and the logic behind the mind map application a decent knowledge of Javascript is advisable, but on the other hand this could be very appealing to node.js developers, as they could immediately compare it with Vert.x. Then, also a client-side Javascript developer can easily follow the content and learn some server-side programming. I'm not sure on my side if I would pick Javascript for developing complex server-side logic, but adopting the same language for both client and server development undoubtedly has its own advantages, as the growing popularity of node.js testifies.

What I especially like in this book is the capacity to explain "full stack" concepts in small, easy steps. You learn about Vert.x and asynchronous message exchanges via the event bus, but also some client side programming techniques in Javascript. The bridge between client and server is built using the SockJS library, which provides real-time, full duplex, client-server communication through HTML5 WebSockets and other fallback mechanisms for older browsers.

Another plus of this book resides in its two last chapters, about modules and deployment. Chapter 5 provides a clear description of distributing and managing Vert.x applications through modules, while Chapter 6 goes into the fundamental topic of real deployment and scaling of Vert.x applications. As Vert.x applications are not distributed as standard Java EE packages, these final chapters are really a bonus and I wish to see more books that include at least some basic information about development, scaling and lifecycle management.


The book is very well written and readable. It is balanced, as it can support both the novice and the expert developer in exploring this excellent Web framework. I can definitely recommend buying it if you want to start studying in deep Vert.x.

Note: I actually bought this book by myself and not asked to review it by Packt or anybody else, so this article reflects my own independent opinion only.


Thursday, June 27, 2013

Apache Maven Starter - my first, little book

Today Packt Publishing has published a little book I have written in the last few months together with Maurizio Pillitu.

The book is titled Apache Maven Starter and it's part of the Instant collection by Packt Publishing. Instant books are concise and focused technical guides, usually no more than 50 or 60 pages, providing the maximum amount of information in the quickest and most effective way.

Get to grips with a new technology, understand what it is and what it can do for you, and then get to work with the most important features and tasks.The book follows a starter approach for using Maven to create and build a new Java application or Web project from scratch.
I accepted the invite to write this book because, despite the huge amount of information already available on this subject, in form of other books and online material, when I started learning Maven I was searching for more concise information. The book actually doesn't aim to be a complete reference, but a place to start from and a collection of pointers to additional information. So, it is the book I wished I could read years ago.
Maven ultimately allows for the automation of the build lifecycle and independence from any IDE. You must always be able to build and test any Java project from the command line, using your favorite editor for coding. It is important to control exactly what libraries get distributed with Java projects and to have a standard project template and build process. 
Instant Apache Maven Starter will concentrate the most useful information into one single, very compact source. 

What you will learn from this book

  • Download, install, and configure Apache Maven with the minimum fuss
  • Make your own Java project templates and reuse them
  • Deploy to Tomcat or run an embedded Tomcat with Maven
  • Perform unit and integration testing with Maven and JUnit
  • Manage dependencies and project coordinates, adopting best practices
  • Create and manage multi-modules projects
  • Use Maven from your favorite IDE: Netbeans, Eclipse, or IDEA

I want to especially thank Maurizio Pillitu and the Packt Publishing editorial team, it has been great to work together guys!

The book is already available on Amazon, Packt and Safari Online.

Thursday, May 30, 2013

Open source products: the big misunderstanding

While reading an article on ZDNet titled "Open source: Its true cost and where it's going awry by Monty Widenius" I think I have realized why most people, even those tightly involved in the open-source ecosystem, are often misunderstanding its business model. In a sentence:
The open source model is about creating collaborations, not building products.
Advocating open source software since the early 90s, from my point of view an open-source business model related to products can be misleading.
"The problem is — I saw this very clearly with MariaDB — I created a company where I took the original people who were creating the product, [but] I couldn't get anybody to fund us," Widenius said.
Before being bought by Sun for an astronomical amount of money (1 billion $), MySQL was profitable but in the range of few tens millions per year. You don't meet every day somebody putting billions on the table...

My question is: what would force a company to pay for a product if they can have it for free? I guess RedHat owners asked themselves the same questions when they practically closed the RHEL product line source code, moving the community toward the Fedora project. And now RedHat is making an huge amount of money from a product that is, by all means, the (almost) closed version of an open source effort.

So, it is really hard to build open source products, like RedHat or Alfresco, for example, because, despite all the efforts to justify a dual licensing model, you cab grasp only a very small percentage of users available to pay for a commercial edition of any open-source software. Stressing the fact that going into production requires paid support can be helpful, but it's very hard to demonstrate where the break-even is. Somebody coud argue: is that product so buggy that I need to pay a lot of money for support? Very few companies have been successful in this.

Instead of selling standalone products, I see an easier path in building solutions and services on top of an open source ecosystem: companies and their developers collaborate in growing a set of tools, frameworks, libraries with an open source development model and communities. This allows to dramatically decrease R&D costs, sharing the benefits. Today I could build a whole enterprise platform with open source software only, from the operating system to middleware and client applications, probably with better quality than a closed counterpart. But, as Widenius seems is learning the hard way, building an open source product and having paying customers is a different story: not impossible, of course, but very hard.
Widenius again: "Open source is successful for OpenStack where you have a consortium of companies that are all putting money into developing it."
Exactly what I mean: join to develop the common parts, share the costs and then compete in selling some unique services. But a stand-alone, open source product, created and supported by a single company, is a different story.

Tuesday, November 13, 2012

RESTful API memo: PUT and POST differences

Before start designing a RESTful API, have a look at Hypertext Transfer Protocol -- HTTP/1.1, section 9

"The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.".

In other terms, POST is meant to handle appends to existing resources or incremental creations of subordinate resources:

"The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database."

PUT instead seems is more appropriate to handle one-shot creations, creating or replacing an entire resource in one single transaction:

"The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."

Differences between PUT and POST:

"The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource."

Another remarkable difference is that PUT requests are required to be idempotent, while POST are not:

"Methods can also have the property of 'idempotence' in that (aside from error or expiration issues) the side-effects of N > 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property. Also, the methods OPTIONS and TRACE SHOULD NOT have side effects, and so are inherently idempotent."

Saturday, November 10, 2012

Creating your private Git repository on Dropbox in less than 5 minutes

Github is the tool I use daily to manage my public software projects, I love it. But sometimes I have to quickly and temporarily share private projects with colleagues or maybe even in a mixed environment, with customers and consultants from other companies. When there is no time / money to buy private remote repos from Github or even install a local Git repo on some server,  and for privacy constraints it is not possibile to publish the code on a public Github repo, then Dropbox comes to the rescue.

In this example I'm working on a simple Web application in Flask, which is a cool Python micro-framework. I created a "flask_sample" folder which contains the code I want to version with Git and share with other colleagues.

I promised it will take less than 5 minutes, so let's start.

Move to your Dropbox folder (in my case it's in /Users/mturatti/Dropbox/) and create a folder to host all your remote git repositories:

$ cd /Users/mturatti/Dropbox/
$ mkdir git

Then create here the folder to host this remote repository:

$ cd git
$ mkdir flask_sample.git
$ cd flask_sample.git

It's time to create a bare Git repository:

$ git init --bare

You'll see it creates a structure similar to the following:

mturatti:~/Dropbox/git/flask_sample.git$ ls -l
total 24
-rw-r--r--   1 mturatti  staff   23  9 Nov 18:38 HEAD
-rw-r--r--   1 mturatti  staff  112  9 Nov 18:38 config
-rw-r--r--   1 mturatti  staff   73  9 Nov 18:38 description
drwxr-xr-x  10 mturatti  staff  340  9 Nov 18:38 hooks
drwxr-xr-x   3 mturatti  staff  102  9 Nov 18:38 info
drwxr-xr-x  11 mturatti  staff  374  9 Nov 19:09 objects
drwxr-xr-x   4 mturatti  staff  136  9 Nov 18:38 refs

Now you have in place a git structure which can act as a shareable remote repository, even if in practice it's local to your hard disk. Being a Dropbox folder will do the magic in terms of backups, sharing and synchronization.

Initialize Git in your software project as usual (in my case the local project stays in /Users/mturatti/src/flask_sample)

$ git init

This creates the usual hidden .git folder.
The last configuration step is to add locally the previously created remote Git repository:

$ git remote add origin file:///Users/mturatti/Dropbox/git/flask_sample.git

Note we are using the file:// protocol for the remote Git repository here.
If you check the content of .git/config file you'll see the new origin (in bold below):

mturatti:~/src/flask_sample$ cat .git/config 

repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
ignorecase = true
precomposeunicode = false
[remote "origin"]
url = file:///Users/mturatti/Dropbox/git/flask_sample.git
fetch = +refs/heads/*:refs/remotes/origin/*

At this point you can start the usual Git lifecycle. For example, after you have added and committed all your files locally, you can "push to origin", which will push your code to your remote Git repository saved on Dropbox:

$ git push origin master

The last step will be to share the Dropbox folder with your colleagues, so that they can also add this as a remote repository and start cloning / pulling / pushing from this origin.

Saturday, September 29, 2012

Time, Cost, Quality and Agile Consulting

Types of Consulting Engagements

“Sometimes it's a little better to travel than to arrive” 
― Robert M. PirsigZen and the Art of Motorcycle Maintenance: An Inquiry Into Values
In my software consulting experience I have been engaged in many different kind of projects, but in all cases they fall into two main categories:

  1. Time and Material (T&M)
  2. Fixed Price (FP)
In most situations it happens than to decide that option #2 is achievable, then a quantity of T&M analysis must be performed in advance, in order to define the context and the scope for a possible, successive FP engagement. That's not always possible: for example, when the project is part of a public tender, you have to bid for the lowest possible price, trying to balance the need for adding a good amount of contingency, staying into the safe path, without self-sabotaging the possibility of winning the tender.

What marks the difference between T&M and fixed price? In T&M a customer is basically paying for your time, because deliverables and scope can't be clearly set in advance, or because it's already established that requirements are going to change in a way that a Fixed Price engagement is out of question, because it's too risky. A Fixed Price project is based on a set of much more stringent assumptions, in terms of context, requirements, functional and technical, which (hopefully) allows for a very accurate estimate of deliverables.

Usually customers are more keen on FP because, of course, they think it will constraint the final price by putting much more responsibilities on the consultant, while T&M seems a way to create a continuos stream of expenses. However, in reality, there is a more fundamental law which regulates any kind of software project, despite the rules of engagement, and it is related to the existing and unavoidable strong relationship among three distinct, fundamental quantities: Time, Cost and Quality.

The Basic Conjecture of Time, Cost and Quality

“When analytic thought, the knife, is applied to experience, something is always killed in the process.” 
― Robert M. PirsigZen and the Art of Motorcycle Maintenance: An Inquiry Into Values
It's almost incredible how many people actually think they can leverage Fixed Price to be in total control, at the same time, of these three quantities:
  1. The elapsed time spent from start to finish, so the final delivery date;
  2. The total cost, in terms of direct money and indirect materials;
  3. The overall quality of the final product.
This assumption has been historically proven false by practice, for any non-trivial software project or consulting engagement. It's a conjecture and not a theorem or a physical law, but reality has taught me that, in software development and software consulting, it is possible to accurately be in control of only two over three of these quantities.

Some examples are needed: if a project has a fixed delivery date and a fixed price, then the only left quantity one can possibly control is quality. Would it maybe explains why so many FP projects suffer from poor perceived quality?
On the other hand there is another fundamental speculation which states that "Nine women can't deliver a baby in one month". It means that, if delivery dates and quality are fixed, one is tempted to keep adding resources, in terms of people and infrastructure, loosing control about costs. In practice this tactic even leads to also increasing delivery time, because adding people on a late project usually delays it even more.
A third case is when we try to fix both cost and quality, but then we accept that elapsed time can't be predicted accurately. This is the case, for example, of companies trying to outsource development to offshore facilities, where cheaper labor force can easily be hired. Statistically this strategy has led many projects to both an indefinite development time but also poorer quality.

It's all about one single truth: software development is inherently not a traditional engineering activity. Actually, Programming is Gardening, not Engineering

Agile Development and Agile Consulting

"Simplicity is the ultimate sophistication". ~ Leonardo da Vinci.
So, at first sight, it seams there is no escape from the T, C & Q rule. But we are not doomed. In fact, as I wrote before, the rule applies for any non trivial software project. So the trick here is: to transform big projects or big consulting engagements into a finite sequence of very focused, well defined, little activities or mini-engagements. This is why time-boxing or feature-boxing usually work effectively, and that's what actually Agile Methodologies are, more or less, trying to achieve: transforming complexity into something more manageable and predictable, by splitting big activities into little, possibly trivial, short tasks, which can be handled in few hours or days by very few people.

I think that agile methodologies can and should be successfully applied also to pure software consulting, so to the kind of engagement usually performed in T&M. The main pillars of this strategy are nothing new and can be summarized as:
  • Focus on User Stories;
  • Short iterations, usually no more than two or three weeks long;
  • Continuos Integration and Continuos Delivery of valuable pieces of software;
  • Acceptance tests at the end of each iteration or when a single deliverable is ready.

As a side note: if you are strictly required to be on-site then that is not a Fixed Price project by definition! Fixed Price engagements MUST be off-site, exactly because you don't want to waste time renegotiating the scope each single moment. It is necessary to have customers involved daily and keep things very flexible, but asynchronous interruptions must be avoided at all costs. The main objective is to understand what final users want and adapt when requirements are changing, but this must be addressed by the process, not individuals. T&M is very different: as customer pays for your time he is entirely entitled to interrupt you and change tasks even in the middle of them. That's why committing on any detailed deliverable in a T&M assignment is extremely dangerous.

The Need for Good Architectural Decisions

"We are searching for some kind of harmony between two intangibles: a form which we have not yet designed and a context which we cannot properly describe." ~ Christopher Alexander.
The missing piece, for a Use Case made of several User Stories, is a comprehensive and reasonably complete Technical Architecture. In other words, I do believe in that kind of bottom up, emerging software design coming from an Agile, iterative process, but I think this must be developed within the frame of a clear up-front Architecture.

I mean that refactoring code and design is not only necessary, but even desirable. Then, in my experience, refactoring wrong initial architectural decisions can be extremely expensive and usually leads to big failures. I strongly believe that the role of an experienced architect is key to produce quality software systems, and this fact sounds to me to be often too underestimated in the field of Agile Methodologies. Do not fool yourself by believing your so-called "rockstar developers" (horrible term!) alone can also actually imagine, design and implement a complete and working technical architecture.

Speaking of consulting, even a short T&M engagement should be performed in the context of a well designed architecture, because even the best expert can be unable to deliver anything useful if the architectural context is broken. Focus your first steps at customer site on two main things: understanding their existing architecture and development process, and start fixing them if they are clearly broken. Otherwise the risk of failing and not get paid will be too high, despite the fact it's T&M or FP.