1. 首页
  2. 问答
  3. 配置单元不允许冒充配置单元

配置单元不允许冒充配置单元

2016-11-15 119 views
2

我有一个配置了hadoop 2.7.2和配置单元2.1.0的hadoop群集。配置单元不允许冒充配置单元

我用直线连接使用下列命令蜂巢：

beeline 
beeline> !connect jdbc:hive2://localhost:10000 
Enter username for jdbc:hive2://localhost:10000:

起初，键入任何用户名可以成功连接到蜂巢。但是，在配置下列设置之后，会显示错误。

//within hive-site.xml and hiveserver2-site.xml 
<property> 
<name>hive.server2.enable.doAs</name> 
<value>true</value> 
<description> 
    Setting this property to true will have HiveServer2 execute 
    Hive operations as the user making the calls to it. 
</description> 
</property> 

//within core-site.xml 
<property> 
<name>hadoop.proxyuser.hive.hosts</name> 
<value>*</value> 
</property> 

<property> 
    <name>hadoop.proxyuser.hive.groups</name> 
    <value>*</value> 
</property>

我已经重新启动Hadoop集群，但仍然出现以下消息：

Error: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive (state=,code=0)

调试输出hiveserver2是如下：

16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: starting, having connections 1 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive got value #-3 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: closing ipc connection to /10.104.90.40:8020: User: hive is not allowed to impersonate hive 
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: closed 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: stopped, remaining connections 0 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: WARN service.CompositeService: Failed to open session 
java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     ... 21 more 
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     ... 28 more 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG security.UserGroupInformation: PrivilegedAction as:hive (auth:PROXY) via hive (auth:SIMPLE) from:org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO session.SessionState: Updating thread name to 74dcfeae-04c1-494d-b491-df53f0d20039 HiveServer2-Handler-Pool: Thread-38 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO session.SessionState: Resetting thread name to HiveServer2-Handler-Pool: Thread-38 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG session.SessionState: Removing resource dir /tmp/hive_resources 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Trying to connect to metastore with URI thrift://10.104.90.40:9083 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Opened a connection to metastore, current connections: 3 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Connected to metastore. 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG metadata.Hive: Closing current thread's connection to Hive Metastore. 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Closed a connection to metastore, current connections: 2 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG ipc.Client: stopping client from cache: [email protected] 
16/11/15 11:28:47 [HiveServer2-Handler-Pool: Thread-38]: WARN thrift.ThriftCLIService: Error opening session: 
org.apache.hive.service.cli.HiveSQLException: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:336) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     ... 13 more 
Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     ... 21 more 
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     ... 28 more

我在网上搜索，发现大部分的解决方案与此错误相关的是actualLy我配置的设置导致问题。

有谁知道如何解决这个问题？

更新：

经过一番尝试，我发现这个错误涉及到哪个用户正在使用的启动蜂巢服务。

在我有用户配置单元之前启动配置单元存储Metastore和hiveserver2导致错误消息。

但是使用用户hadoop这是启动hadoop namenode来启动hiveserver2的用户摆脱错误消息。

我没有弄清楚为什么使用不同的用户是一个临时解决方法。

来源

2016-11-15 Heyang Wang

+0

为什么你想用特定的用户开始Metastore（为什么不只是hadoop？）。 - 只是猜测，但也许启动Metastore的用户定义了配置/日志的放置位置。而对于一些用户来说，这些位置可能不存在或可能不够容易访问。 –

+0

起初，这是因为大多数这些在线教程都使用配置单元作为用户。但后来，我想可能是所有的蜂巢服务使用蜂巢服务是不必要的sice蜂巢用户只具有蜂巢文件夹的特权，它不能搞乱hadoop或spark文件夹。 –

+0

也许我从您的评论中得出了错误的结论，但如果您担心访问错误的文件夹，我建议您只使用“标准”用户帐户进行初始设置，然后使用自定义用户帐户。 （像王和管理王）。 –

回答

0

我也遇到类似的问题。这是我为了使它工作而做的。 在蜂巢-site.xml中，你有这样的特性：

<property> 
     <name>hive.conf.restricted.list</name> 
     <value>hive.security.authenticator.manager,hive.security.authorization.manager,**hive.users.in.admin.role**</value> 
    </property>

我只是删除了部分大胆上面所强调的：hive.users.in.admin.role。

它开始为我工作。

来源

2017-02-08 07:22:41

相关问题

 相关问题