Buidling consumer analytics - google way

The recent IBM Watson Jeopardy game show is one of the iconic moments in IBM’s history. At least the event has showcased and evangelized that way.  No doubt, truly it’s amazing game show and provides an insight the way problems can be solved in real time by computers and definitely a technology of future. I am not sure whether there will be any dot in Gartner’s hype cycle.

As per IBM press, the technology can be adapted in real life use cases like in health care, transports,  utility, weather  forecast, natural calamities. This technology is definitely is one of the important catalyst to offer the big blue’s smart planet vision. IBM is already spending billions of dollar in business analytics and lined up an impressive stack of Analytics software to solve any kind of customer analytics problem. The Watson project is one such commitment from IBM to add on.

On the contrary, Google has dominated over the web and its roots are from search. The Watson’s technology is also based on semantic and meaning based search may be they use different algorithms.   There are many companies tried to solve meaning based search and this technology already exists before. For example Wolframalpha, Powerset use semantic based search.

 In principle the index based search and meaning/context based semantic search are very different.  You also need to think of processing time.  Typical search engine's response time is very very less( fraction of seconds ). So it would not be really important for Google as it might not perceive much of business values by adding the semantic based search. Well who knows in future Google might build it with the general search. To make it happen Google needs to invest, but I do not think it is very difficult for Google to achieve this.

I think so far Google core strength is in consumer side and also in the web. If you watch their technology and products built, they are purely built on search and analytics where it entirely leverages their mammoth of data discovery and indices which are used for Google search.

Projects like Google Trend, Google public data explorer are based on search and aggregating data and build analytics on top that which is a real importance to customer.  Most of the Google products are in line with consumer analytics. As government and citizen seek more transparency, lots of public data will be available in the web and they will be more unstructured. In addition to that the social networks will provide the impetus to consumer analytics. Already lot of vendors capitalizing on the social data for analyzing different consumer trends.  The customer experience and sentimental analysis is one of such kind of implementation.

In current context the business trends not only evaluated based on organizational data but also the analysis of web and social data is equally important. What I mean is for an enterprise to take effective decisions, it should analyze the enterprise data and social data in order to get a 360 degree view. This is what precisely “Consumer Analytics”. For this to happen we need to synthesize the social net and I think Google has first mover advantage in this space.

Better know your NoSQL before leaving SQL

The NoSQL momentum is gaining rapidly. Most of the vendors embracing this new data model. However, there is  lot of misconception and hype is built surrounding this. This article provides few pointers for initial understanding of this new breed of data base.

1.  Is NoSQL is the replacement for SQL

Yes, if you say "YES" to following :
a.Is my data is unstructured ?
b.Do my data access patterns lend themselves to demoralization ?
c.Do I expect lot of schema change in my app ?
d.Can I Drop ACID  properties ?
e.Do I have more complex joins ?

2.  Is  NoSQL means Map Reduce ?
NoSQL is an implementation of a specific type of Data storage Model.  Map reduce is programmatic models to process large data sets. Most of the NoSQL databases offer some kind Map Reduce feature/implementation. That does not mean that we need to necessarily use map reduce to store data. Map reduce is used to extract and aggregate data at large scale from underlined data stores.   So NoSQL is purely a Data Storage platform in other hand Map Reduce is a way to process data.

3.  Can I have both SQL and NoSql  living and breathing  together ?
We can have both SQL based Data Store and NoSQL based data store living together. In most of the cases they will be complementary to each other as they try to solve different problems.

4.  Is NoSQL only for BigData
Truly NoSQL is not merely meant to solve the BigData issue. NoSQL is more popular in BigData use cases as many vendors solved their large scale data problems implementing  NoSQL databases.  NoSQL architecture naturally lends itself to solve today’s Big Data problem. There are lot of vendors still use Relational data store for  their apps. Face Book uses MySql as part of their data base layer. Most of the future apps will be built on polyglot persistency where the same app is built on multiple data models one could be relational and another could be a key and value based NoSQL database.

5.  Which is better performer SQL or NoSQL
The performance of app depends on many factors. We cannot be thinking using NoSQL data store can bring magic and solve all our pains over night. NoSQL is good for some kind of apps which are described above and SQL is good for different kind of apps. Use the data store which is best for solving a specific kind if issue. If I have large no of atom feeds  with different schemas and would not like to store in relational database would be a big pain. Select write data store for a specific  problem. Do not follow the hype of NoSQL use your analytics. NoSQL definitely cannot solve all kinds of problems. For more to understand Horizontal Partition versus Vertical Partitioning go thru (http://venkat-sp.blogspot.com/2009/12/scale-up-versus-scale-out-scale-up-this.html )

6.  Is NoSQL means more CPUs with large scale projects
Not really. I can run MongoDB, Cassandra , Redis in single system. But All these data stores architectures allows data to be partitioned and scale horizontally. In other terms we can have many shards as data grows. This is one of the important feature of any NoSQL databases available in the market and precisely this is the reason why NoSQL DBs are making in roads in big way for scale out problems and majorly seen in Scalable Apps.

7.  Is NoSQL means cloud DB
Most of the cloud vendors store data using some form key values or document kind of data store which helps them to scale out easily. That should not necessarily conclude NoSQL means cloud DBs. Amazon provides Mysql instances ( http://aws.amazon.com/rds/ )  for its customers. Very recently Amazon also support  Oracle instances ( http://aws.amazon.com/rds/oracle/ ). These all are Relational databases.

Please add you comments. Thanks.

Does JavaFX2 has life ahead which has released recently with a complete overhaul ?

Oralce has released Javafx 2.0 Rich Client Access ( Client side RIA ) beta. The JavaFX 2 plugin support is given in the new Netbeans 7.

The Major feature includes:

New Improved Java APIs for JavaFX : 

1. Allows to work with the java heritage

2. Support for DSLs

3. Swing Interoperability: No alien for java swing developers

4. A prism capable plugin: A plugin capable of running prism without starting of AWT.

5. A new graphics stack with new fully hardware accelerated pipeline called Prism targets DirectX on windows and Opengl on other platforms.

6. A web component that uses Webkit to render HTML.

7. An ultra net set of UI controls

8. New media features like Alpha Channels , Audio Equalizer, Low latency audio and programmatic Media Marker.

Find more details (http://javafx.com/roadmap/ )

Java is very strong and is more prominent in server side. It has not seen much success in client side. Sun developed very innovative client technologies like swing and applet sticking to the tag line

“write once run anywhere”.

Till today the applets are in use in many production environments. The java + opengl combination in the client side makes more appealing and as applet runs on the browser, it makes much more important the role of applets in browser to display the media content. There are several other alternatives technologies have been emerged using opengl but the good old way of displaying 3D content would be using a simple applet.

But somehow all these innovations failed to attract building client side applications using java. Till today sun/oracle did not bridge that gap may be it is mainly because an enterprise cultural issue or lack of focus. On the contrary Microsoft always excelled in providing nice UI friendly languages be it VB or C# with window forms and wpf. In the browser side it has very strong product to beat the competition is Silver Light. So developers are comfortable developing .Net stack the natural choice of the RIA platform is Silver Light.  

On the same page while Adobe had some strong comments from Steve Jobs on using flash in apples product line, but Flex is one of the most used technology in the client side. ( 90%) web applications use flash in their websites. That itself shows the product acceptance.

The following picture shows the job trends between competing products. javafx, extjs, adobe flex,silver light.

  

   

While all this is fine, there is new kid in the block which is HTML 5. Most of the companies are betting on HTML 5 to be the defacto standard for client side apps such as displaying 2D and 3D contents, audio/video stream to building client side apps with JS being the underlined language. Microsoft has recently raised confusion on Silver light’s feature as it believes the HTML is way feature and so apple.

The biggest question is do we have any need for any new RIAs with new learning curve and new nuances.

I feel each RIA vendors embrace HTML as standard platform, still these technologies will have their own approach to make HTML5 work for app developers. 

Let me know your comments.

Greenplum : The MPP ( Massively Parllel Processing ) architecture explained

Greenplum is a well know Database used for very large scale analytics and its Massively Parallel Processing architecture based on postgres sql database. It has a nice client list and deployments are mostly in tera to petabytes range of data. That speaks for itself. The following blog explains the architecture details of Greenplum which is recently aquired by EMC.

Orginal Post :

http://patodirahul.blogspot.com/2011/04/understanding-greenplum.html

Embedding flex apps with in PDF documents ( Portable RIA )

The following is the nice post which explains how to embed Flex apps with in PDF documents. James Ward Technical evangelist for Flex at Adobe has put a nice post on this which he calls as portable RIA.

http://www.jamesward.com/2008/11/05/portable-rias-flex-apps-in-pdfs/

  

Document Databases Compared: MongoDB, CouchDB, and RavenDB

Please find the Document database compare between MongoDB, cuchDB , RavenDB

http://nosql.mypopescu.com/post/613939499/new-projects-in-nosql-space

Choosing the right PAAS vendor for your application

The entire cloud ecosystem is getting too hot and crowded with various language and tool support to enable the common application to be ported easily to the cloud . If you search thru google then you find many PAAS player and they claim they are the best with lot of testimonies. Almost all popular languages are supported by most of the cloud vendors.

Notably the most popular PAAS platforms are

1. Force.com from Sales force
2. Windows Azure from Microsoft.
3. App engine from GOOGLE.
4. vFabric from VMWare

The factors influence during selection of PAAS platforms are following:

1. Application type and complexity.
2. Project Type ( Is it a brand new project or old project want to migrate to cloud )
3. Preferred Technology Stack
4. Allocated Budget
5. Business goals
6. . Project time line
7. Developers competency
8. Vendor lock in

Force.com is the leading PAAS offering for building business application from Sales force which is kind of ready to use recipe from Sales force. Simple business applications can be created with no time using Force platform.

It has a visual platform, in which the users can drag and drop different objects to construct the entire business work flow. This work flow can be readily deployed over cloud using the Force platform. There are thousands of applications using this platform. To write complex logics Force provides apex scripting for writing business logic.

Very recently Sales force upgraded the whole force platform and offers different products targeted on Force platform.

1. Appforce : For building enterprise Apps

2. Siteforce : For building data-rich web sites

3. VMforce : Building java apps

Force.com platform initially targeted towards the business users and who are not really keen on learning new languages and the most of the application built are business applications. It is easy to build and easy to host and hence attracted lot of users.

We might see some momentum in enterprise applications hosted on Force.com platform as sales force is heavily investing on this platform and continuously enriching the platform to support multiple customer’s needs.

Windows Azure is a full stack PAAS platform tries to provide end to end cloud deployment. Being it is product from Microsoft very much entitled for “Enterprise Ready Platform”.

Microsoft has tried its best to provide rich tool set for building cloud application using .Net framework. So it is very natural fit for vendors who have been already developing application using .Net.

Azure is one of the most rich and mature PAAS platform available for the developers for different type applications.

1. Azure offers two different kinds of application models: Web Roles and Worker Roles. Web roles are typical request/response HTTP model where as worker role is a generalized development model which can run in back ground.

2. Developers can use array of languages like C#, VB.Net, PHP ,Java for developing applications on .Net framework.

3. Visual studio IDE is available for building and deploying applications with ease. Most of the complex tasks like build to deploy to cloud can be done easily using simple clicks.

4. It has support for Relational database using conventional ADO.NET service and also NoSql database.

5. Azure Appfabric service bus provides secure messaging and connectivity capabilities that enable building distributed and disconnected applications in the cloud.

The above features makes it as a natural fit for existing .Net applications looking for migrating to cloud.

Google App engine ( GAE ): Google also offers all the major features provided in other PAAS platforms. Google supports very few languages like Java and python. For java developers there is very limited support on server side java but the support is pretty stable and mature.

On storage side it provides key value based no sql storage type on its proprietary Big Table database with auto-partitioning. When you consider the scalability factor, GAE is the winner as it scales automatically depends the need.

GAE platform offers one stop solutions for python users. It supports python with DJango popular web frame work. Pretty soon Google will definitely offer the entire java enterprise stack as that will be the future for GAE. There are lot of other vendors provide different tooling and web frame works on GAE to accelerate and minimize the pain involved in cloud migration.

Google also provides necessary tools for building cloud application using GAE on existing eclipsed IDE which is again favors GAE adaption.

vFabric : VFrabic is from VMWare and is a late entrant to cloud offering. Traditionally VMWare has its origin from Virtualization and virtualization is at the heart of cloud computing. Hence it is very natural for VMWare to have cloud offering. After it acquired Spring source has built the entire cloud ecosystem around it’s technology needed to push the much needed cloud envelope.

vFabric supports only java as vFabric is for purely 100% Spring Java development framework. Spring tool suite provides the best eclipsed powered development environment for building spring base applications. vFabric also supports groovy and grails as part of its offering. This is a good value proposition for vendors already using J2EE and spring frameworks for their enterprise applications. They can move their application to cloud very easily using vFabric Enterprise ready server.