Convert JSON data of Localytics to CSV using Python

Extraction Transformation Loading (ETL) is very important at the inception of data analytics. Data could come from variety of sources and formats. Either you have to adopt as it is or you have to convert into one common format. Here in this demo we will go through second option.

For those who do not know much about Localytics. Localytics is a lifecycle engagement platform for web and mobile apps. Their integrated approach to app marketing and analytics helps customers automate and optimize every stage of the app lifecycle to keep their users engaged and to deliver more personalized experiences. Localytics is used in more than 37,000 apps on more than 2.7 billion devices by companies such as ESPN, eBay, Fox and The New York Times. Localytics is headquartered in Boston, MA with offices in San Francisco and London. Investors include Sapphire Ventures, Foundation Capital and Polaris Partners. Read More Here.

As a part of their enterprise license Localytics does provide an API to access the raw data of your app analytics. Once you subscribe their Enterprise service you will get HOW TO guide by email. This guide contains how to access raw data from amazon s3.

Localytics exports raw data in JSON format. For IT professionals and MongoDB experts JSON is like their devotion. However for Data Analysts and Modelers it can be headache. Both of them can be either Excel or SPSS lovers who spend their time with tabular data.

As a part of this process We have created quick python script that will covert the json file to csv format.

Run this script from command line using,

Script will create a csv file with the same name in the same directory from which you are executing it. Happy Coding

 

Auto Complete with Redis in Java

I hope the topic of this post will be very interesting for many Redis users, and for many non Redis users that experimented the pain of modeling auto complete for web or mobile applications using a database that is not designed to make this a simple task.

Auto complete is a cool and useful feature for users, but it can be non trivial to implement it with good performances when the set of completions is big.

Simple auto complete

In the simplest auto completion schema you have:

  • A set of N words, that are in some way notable words that you want to auto complete. This can be for instance the names of the biggest cities in the world in a subscription form, or the most frequent search terms in a search service. We’ll try to complete a list of female names. You can find the raw file name here: Names.txt.
  • A prefix string, that is, what your user is writing in the input field.

Given the prefix, you want to extract a few words that start with the specified prefix.

But in what order? In many applications (like the list of cities in a web form) this order can just be lexicographic, and this is more simple and memory efficient to model in Redis, so we’ll start with it, trying to evolve our approach to reach more interesting results.

Let’s start implementing this in Java. Before starting I will assume that you have already downloaed & installed redis-server on your server.

First of all we need a Java client which will provide API to easily interact with the redis server. There are many Java Clients available on redis site. We will use Jedis (A blazingly small and sane redis java client); Download jar from here and add it in your projects build path.

That’s all. Crunching Million records in few seconds. Stay tuned I will be soon posting full featured example of Redis. Thanks for reading.

Happy Coding.