Apache Hadoop monitoring

[suresh230591]

Hello,

I have some questions/doubts about hadoop cluster monitoring . Can someone help me to understand the following

1) How to see the hadoop cluster specific for specific node. I can't see "NameNode" metrics on NameNode process page like I see for DataNode.However I could see these metrics under technology specific metrics of process group "apache.hdfs.NameNode".

2) Could it be possible to pin the "Custer Chart" from technology specific metric of NameNode process group to dashboard.

3) Could not find NameNode metrics in the custom charts metric list , however I could see those under technology specific metrics of process group "apache.hdfs.NameNode"
for example : CapacityCache capacity in bytes

[Jakub_Mierzewsk]

1) Hadoop cluster data are avaialve per cluster not per node
2 and 3) Currently cluster charts metrics (metrics bind to process groups) are not supported in custom charts and dashboards

[suresh230591]

@Jakub M.

We have clustersfor hadoop, Spark, Hive and kafka. If I look at NameNode metrics, it shows the cluster chart metrics. How do I know that these metrics are for which cluster ( Spark , hadoop , hive)

Same applies for Yarn Resource Manager

You can see , we have 6 PGI of name node.

Also NodeMangager does not show any cluster data, however it show data on PGI level

[vladislav_samoy]

Good question