12.5.15

Efficiency Comparison of All Possible Ways to Dump Database (mysql) into HDFS


  • sqoop + mysql connector
    • It use mysqldump command to do the job, which makes it the most efficient solution.
  • sqoop + mysql jdbc driver
  • MapReduce + jdbc driver
    • It is good but less efficient than sqoop + mysql jdbc driver because sqoop will optimize the job.
  • mysqldump to local file system than copy to hdfs
    • overhead of storing the data twice.

No comments: