LOAD and STORE from HIVE Table to PIG

5 comments

We might face a scenario or use case where we might be asked to perform Data crunching or munging by loading the data from Hive table into Pig Relation and after performing the Data analysis or Data Transformation using PIG we might be writing it back to again HIVE Table, Here we are going to practically work out this scenario.

Class Room Video

First Let us consider that we have the Hive table called student with some data in it.

Let us also take another table called College with out any data and it should be with only the schema details.

We nee to start the Pig Latin command shell using the following command on your shell prompt.

Don’t forget to use  -useHCatalog


LOAD Data from  HIVE Table into PIG Relation

org.apache.hive.hcatalog.pig.HCataLoader(); will load the data from HIVE Table into PIG Relation called a1;

Now the a1 relation has all the data, Let us try to filter only the values where std_id =102


STORE Data from  PIG Relation Into HIVE Table

Let us now write back the above a2 Pig Relation back to HIVE Table.

Let us check whether the college table has now only one row.


Store PIG Relation into HDFS

We can also store the Hive table data that has been procesed as Pig Relation into HDFS.

 Output in HDFS:


How to load  all the files from a directory into Pig Relation without Schema?

  • a1 is the relation with out schema.
  • Since /user/cloudera is a default folder in hdfs, we can use folder name directly.
  • using is the keyword which may or mayn’t be case sensitive but PigStorage or any function should be Case sensitive otherwise it will throw an error.
  • all the statements in pig should end with ;(semicolon)

How to load all the data into Pig Relation with schema?

  • though i have used a1 as relation name, i can still use A1 with capital letter both will act as a 2 different relations.
  • now A1 relation will have schema with year, dayofmonth, arrdelay, dest…

Let us practice this using Pig Shell command prompt.

Without Schema details

With Schema details

Leave a comment

Your email address will not be published. Required fields are marked *

5 thoughts on “LOAD and STORE from HIVE Table to PIG”