Sqoop นำเข้าจาก MySQL ไปยัง Hive การโยนข้อผิดพลาด HIVE_CONF_DIR


0

ฉันพยายามโหลด MySQL tableLab5 ไปยัง Hive โดยใช้ Sqoop Sqoop วิ่งได้สำเร็จและนำข้อมูลจาก mysql ไปสู่ ​​hdfs แต่การโยนข้อผิดพลาดตัวหนาด้านล่างเกี่ยวกับการHIVE_CONF_DIRไม่ได้ตั้งค่า ฉันได้ตั้งค่าHIVE_CONF_DIRใน~/.bashrcและ~/.profileเป็น:

export HIVE_HOME=/usr/local/hive
export HIVE_CONF_DIR=$HIVE_HOME/conf

ทั้งๆที่ฉันกำลังเผชิญกับHIVE_CONF_DIRข้อผิดพลาดด้านล่าง คุณกรุณาช่วยฉันด้วยข้อผิดพลาดนี้โดยเร็ว?

18/07/16 13:27:38 **INFO mapreduce.Job: Job job_1531743854195_0001 completed successfully**
18/07/16 13:27:38 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=566628
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=408
        HDFS: Number of bytes written=179248
        HDFS: Number of read operations=16
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=8
    Job Counters 
        Launched map tasks=4
        Other local map tasks=4
        Total time spent by all maps in occupied slots (ms)=12437
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=12437
        Total vcore-milliseconds taken by all map tasks=12437
        Total megabyte-milliseconds taken by all map tasks=12735488
    Map-Reduce Framework
        Map input records=1000
        Map output records=1000
        Input split bytes=408
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=338
        CPU time spent (ms)=3300
        Physical memory (bytes) snapshot=713752576
        Virtual memory (bytes) snapshot=7925747712
        Total committed heap usage (bytes)=458752000
    File Input Format Counters 
        Bytes Read=0
    File Output Format Counters 
        Bytes Written=179248
18/07/16 13:27:38 INFO mapreduce.ImportJobBase: Transferred 175.0469 KB in 18.8594 seconds (9.2817 KB/sec)
18/07/16 13:27:38 INFO mapreduce.ImportJobBase: Retrieved 1000 records.
18/07/16 13:27:38 INFO mapreduce.ImportJobBase: Publishing Hive/Hcat import job data to Listeners for table tableLab5
Mon Jul 16 13:27:38 IST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
18/07/16 13:27:38 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tableLab5` AS t LIMIT 1
18/07/16 13:27:38 INFO hive.HiveImport: Loading uploaded data into Hive
18/07/16 13:27:38 **ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.**
18/07/16 13:27:38 ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
    at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
    at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:392)
    at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:379)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:337)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:44)
    ... 12 more

คำตอบ:


0

คุณช่วยลองส่งออก hadoop_classpath แบบนี้ได้ไหม

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/lib/hive/lib/*

แล้วก็

source ~/.bashrc
โดยการใช้ไซต์ของเรา หมายความว่าคุณได้อ่านและทำความเข้าใจนโยบายคุกกี้และนโยบายความเป็นส่วนตัวของเราแล้ว
Licensed under cc by-sa 3.0 with attribution required.