Colin McCabe (Cloudera)
Apache HTrace is a new open-source end-to-end tracing framework designed to address the challenge of distributed tracing.
HTrace grew out of our frustration with diagnosing Hadoop performance problems. As a distributed system, Hadoop's performance is dependent on the performance of multiple networked nodes. To make matters worse, most deployments of Hadoop depend on a group of distributed systems working together-- such as Hive, HBase, and HDFS.
HTrace makes Hadoop performance easier to understand by tracing requests all the way through the system. Tracing can be enabled at the granularity of an individual request. HTrace is also design to be deployed in production, by sampling a subset of requests. In this talk, I'll discuss the design of HTrace, our progress at integrating it into Hadoop and building the community, and give a demo of the new web interface.
