I'm looking for the best way to load incremental data(based on timestamp of source table) into my table. I have a source table(in hbase) where in data gets updated everyday. In the first flow, I will have to transfer the full data into my test table(in hbase). The next day , I need to transfer only the newly added records from the source table. For this I'll be using the timestamp to differentiate what needs to be transferred and what not. So which is the best way to transfer. Should I use PIG, MapReduce or Spark?
