HQL错误：cannot be cast to org.apache.hadoop.hive.ql.exec.persistence.MapJoinTableContainerDirectAccess

最新推荐文章于 2024-08-26 17:39:41 发布

IT小强哥

最新推荐文章于 2024-08-26 17:39:41 发布

阅读量2.9k

点赞数 5

CC 4.0 BY-SA版权

分类专栏： Hive azkaban 文章标签： hive

本文链接：https://blog.csdn.net/jackfeng86/article/details/124703672

Hive 同时被 2 个专栏收录

11 篇文章

订阅专栏

azkaban

5 篇文章

订阅专栏

本文讲述了在使用Azkaban调度Hive作业时遇到的ClassCastException错误，焦点在于空hashtable导致的MapJoin问题。作者提供了设置hive.mapjoin.optimized.hashtable=false来解决该问题的方法，并附上详细报错堆栈信息。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

背景：

新写的azkaban调度，手动执行没问题，定时执行就报错，如下：

11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO - Job failed with java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper cannot be cast to org.apache.hadoop.hive.ql.exec.persistence.MapJoinTableContainerDirectAccess
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO - 2022-05-11 06:13:14,838 ERROR [cc03d3bf-accd-4779-b30b-613ec97f72f8 main] status.SparkJobMonitor (SessionState.java:printError(1250)) - Job failed with java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper cannot be cast to org.apache.hadoop.hive.ql.exec.persistence.MapJoinTableContainerDirectAccess
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO - java.util.concurrent.ExecutionException: Exception thrown by job
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at org.apache.spark.JavaFutureActionWrapper.getImpl(FutureAction.scala:282)
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at org.apache.spark.JavaFutureActionWrapper.get(FutureAction.scala:287)
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:382)
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:343)
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
11-05-2022 06:13:14 CST dwt_to_ads_bi_user_daily_active INFO -     at java.lang.Thread.run(Thread.java:748)

手动执行hql，结果都能出来。