cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Apache Hadoop monitoring

suresh230591
Advisor

Hello,


I have some questions/doubts about hadoop cluster monitoring . Can someone help me to understand the following


1) How to see the hadoop cluster specific for specific node. I can't see "NameNode" metrics on NameNode process page like I see for DataNode.However I could see these metrics under technology specific metrics of process group "apache.hdfs.NameNode".


2) Could it be possible to pin the "Custer Chart" from technology specific metric of NameNode process group to dashboard.


3) Could not find NameNode metrics in the custom charts metric list , however I could see those under technology specific metrics of process group "apache.hdfs.NameNode"
for example : CapacityCache capacity in bytes

3 REPLIES 3

Jakub_Mierzewsk
Inactive

1) Hadoop cluster data are avaialve per cluster not per node
2 and 3) Currently cluster charts metrics (metrics bind to process groups) are not supported in custom charts and dashboards

suresh230591
Advisor

@Jakub M.

We have clustersfor hadoop, Spark, Hive and kafka. If I look at NameNode metrics, it shows the cluster chart metrics. How do I know that these metrics are for which cluster ( Spark , hadoop , hive)

Same applies for Yarn Resource Manager

You can see , we have 6 PGI of name node.

Also NodeMangager does not show any cluster data, however it show data on PGI level


vladislav_samoy
Inactive

Good question