Yarn / Tez / Google protocol buffer error

Some people seem to get errors when executing hive on tez. I saw the same stack trace in several posts:

Searching for details about this error in the hadoop 2.3.0 source code I couldn’t find the relevant classes in the stack trace. The trick is, that the class

org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto

is auto generated during the hadoop build.

mvn install -DskipTests=true -Dmaven.javadoc.skip=true

After the build is finished a search for this class is successful:

The lines that making the trouble are

Searching for this method in internet I found the post in http://goo.gl/s4jmje

Using classfinder we can search all projects on our server for this class and find some relevant matches in several hive projects.

In the snapshot 0.14 build the explicite depdency to the google protocol buffer jar with version 2.4.1 is removed. After switching hive to this snapshot version the error in combination with tez disappears.

Leave a Reply

Your email address will not be published. Required fields are marked *

This blog is kept spam free by WP-SpamFree.