Hive Installation
Download
woir@woir-VirtualBox:/tmp$ wget http://archive.apache.org/dist/hive/hive-2.1.0/apache-hive-2.1.0-bin.tar.gz woir@woir-VirtualBox:/tmp$ tar xvzf apache-hive-2.1.0-bin.tar.gz -C /home/woir
copy paste following in the /home/woir/sourceme
export HIVE_HOME=/home/woir/apache-hive-2.1.0-bin export HIVE_CONF_DIR=/home/woir/apache-hive-2.1.0-bin/conf export PATH=$HIVE_HOME/bin:$PATH export CLASSPATH=$CLASSPATH:/home/woir/hadoop-2.6.0/lib/*:. export CLASSPATH=$CLASSPATH:/home/woir/apache-hive-2.1.0-bin/lib/*:. export HADOOP_HOME=~/hadoop-2.6.0
hduser@laptop:/home/woir/apache-hive-2.1.1-bin$ source ~/.bashrc
$ echo $HADOOP_HOME
/home/woir/hadoop-2.6.0
$ hive --version
Hive 2.1.0
Subversion git://jcamachguezrMBP/Users/jcamachorodriguez/src/workspaces /hive/HIVE-release2/hive -r 9265bc24d75ac945bde9ce1a0999fddd8f2aae29
Compiled by jcamachorodriguez on Fri Jun 17 01:03:25 BST 2016
From source with checksum 1f896b8fae57fbd29b047d6d67b75f3c
woir@woir-VirtualBox:~$ hadoop dfs -ls / drwxr-xr-x - hduser supergroup 0 2016-11-23 11:17 /hbase drwx------ - hduser supergroup 0 2016-11-18 16:04 /tmp drwxr-xr-x - hduser supergroup 0 2016-11-18 09:13 /user woir@woir-VirtualBox:~$ hadoop dfs -mkdir -p /user/hive/warehouse woir@woir-VirtualBox:~$ hadoop dfs -mkdir /tmp woir@woir-VirtualBox:~$ hadoop dfs -chmod g+w /tmp
woir@woir-VirtualBox:~$ hadoop dfs -chmod g+w /user/hive/warehouse woir@woir-VirtualBox:~$ hadoop dfs -ls / drwxr-xr-x - hduser supergroup 0 2016-11-23 11:17 /hbase drwx-w---- - hduser supergroup 0 2016-11-18 16:04 /tmp drwxr-xr-x - hduser supergroup 0 2016-11-23 17:18 /user woir@woir-VirtualBox:~$ hadoop dfs -ls /user drwxr-xr-x - hduser supergroup 0 2016-11-18 23:17 /user/hduser drwxr-xr-x - hduser supergroup 0 2016-11-23 17:18 /user/hive
woir@woir-VirtualBox:~$ cd $HIVE_HOME/conf
woir@woir-VirtualBox:~/home/woir/apache-hive-2.1.0-bin/conf$ vi hive-env.sh.template
# Set HADOOP_HOME to point to a specific hadoop install directory
# HADOOP_HOME=${bin}/../../hadoop
# Hive Configuration Directory can be controlled by:
# export HIVE_CONF_DIR=/home/woir/apache-hive-2.1.0-bin/conf
hduser@laptop:/home/woir/apache-hive-2.1.0-bin/conf$ cp hive-env.sh.template hive-env.sh
export HADOOP_HOME=/home/woir/hadoop-2.6.0
$ cd /tmp $ wget http://archive.apache.org/dist/db/derby/db-derby-10.13.1.1/db-derby-10.13.1.1-bin.tar.gz $ tar xvzf db-derby-10.13.1.1-bin.tar.gz -C /home/woir
copy paste following in the /home/woir/sourceme
export DERBY_HOME=/home/woir/db-derby-10.13.1.1-bin export PATH=$PATH:$DERBY_HOME/bin export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar
$ mkdir $DERBY_HOME/data
hduser@laptop:~$ cd $HIVE_HOME/conf create a new file hive-site.xml woir@woir-VirtualBox:/home/woir/apache-hive-2.1.1-bin/conf$ vi hive-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:derby:;databaseName=/home/woir/apache-hive-2.1.0-bin/metastore_db;create=true</value> <description> JDBC connect string for a JDBC metastore. To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL. For example, jdbc:postgresql://myhost/db?ssl=true for postgres database. </description> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> <description>location of default database for the warehouse</description> </property> <property> <name>hive.metastore.uris</name> <value/> <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.apache.derby.jdbc.EmbeddedDriver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.PersistenceManagerFactoryClass</name> <value>org.datanucleus.api.jdo.JDOPersistenceManagerFactory</value> <description>class implementing the jdo persistence</description> </property> </configuration>
Metastore schema initialisation
woir@woir-VirtualBox:/home/woir/apache-hive-2.1.0-bin/bin$ schematool -initSchema -dbType derby SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/woir/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/woir/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:derby:;databaseName=/home/woir/apache-hive-2.1.0-bin/metastore_db;create=true Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver Metastore connection User: APP Starting metastore schema initialization to 2.1.0 Initialization script hive-schema-2.1.0.derby.sql Initialization script completed schemaTool completed
Launch Hive
woir@woir-VirtualBox:/home/woir/apache-hive-2.1.0-bin/bin$ hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/woir/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/woir/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in jar:file:/home/woir/apache-hive-2.1.0-bin/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive>