Json parsing with Hive

Leave a comment

In this blog, we will see how to parse Json data in hive and perform sql queries on top of this data.Copy and paste below json data and save it as bloger.json:Move data into hdfs using below command.hdfs dfs -put bloger.json /tmp/sample/Log into hive and create table blogCREATE EXTERNAL TABLE blog(value STRING);Load data into table blog from hdfsload data inpath ‘/tmp/sample/’ into table blog;Now, data is loaded into hive. Use below queries to access the json data in tabular format.

Learn More

Hadoop Storage Calculation

Leave a comment

One of the challenge to start with big data projects is to identify the cluster requirements. In this post, we will discuss about calculating cluster size based on (application) data. Below are the important factors for calculatingAverage Compression Ratio(c) : Default value is 1, if we want to store data without any compression. If we would like to store data with compression, below is the formula (1- (Compressed Size / UnCompressed Size) )Example: Let’s assume size of the data without

Learn More

Scala Lists

Leave a comment

5 ways to create List:We can create a Scala List in several different ways, including these approaches:How to merge Scala Lists?There are at least 3 ways to merge Scala lists.

Learn More

Scala Basics

Leave a comment

Scala supports REPL (Read Evaluate Print Loop) interactive shell TuplesBlocking StatementsConditional OperatorsLoopsYieldWhileFunctionsPassing multiple valuesRecursive functions

Learn More

Tags