Skip to main content

Application Performance Monitoring (APM)


1 - What is APM?

1.1 Definition

APM = Real-time monitoring of application performance.

1.2 Key metrics

MetricDescriptionTarget
ApdexSatisfaction score> 0.9
Response TimeResponse time< 200ms
ThroughputRequests/secVariable
Error Rate% of errors< 1%
CPU/MemoryResources< 80%

2 - APM Solutions

2.1 Comparison

SolutionTypePriceStrengths
DatadogSaaS$$$All-in-one
New RelicSaaS$$$Native APM
DynatraceSaaS$$$$AI-powered
Elastic APMOSS/SaaS$-$$ELK integration
GrafanaOSS$LGTM stack
SigNozOSSFreeOpenTelemetry

2.2 Open source architecture


3 - Elastic APM

3.1 Agent installation

// Node.js - apm.js
const apm = require('elastic-apm-node').start({
serviceName: 'my-service',
serverUrl: 'http://apm-server:8200',
environment: 'production',
captureBody: 'all',
transactionSampleRate: 1.0,
});

module.exports = apm;
# Python
import elasticapm

app = Flask(__name__)
apm = ElasticAPM(app,
service_name='my-service',
server_url='http://apm-server:8200'
)

3.2 APM Server configuration

# docker-compose.yml
apm-server:
image: docker.elastic.co/apm/apm-server:8.11.0
ports:
- "8200:8200"
environment:
- output.elasticsearch.hosts=["elasticsearch:9200"]
- apm-server.kibana.enabled=true
- apm-server.kibana.host=kibana:5601

3.3 Custom Transactions

const apm = require('./apm');

// Manual transaction
const transaction = apm.startTransaction('process-order', 'custom');

// Span
const span = apm.startSpan('database-query');
await db.query('SELECT * FROM orders');
span.end();

// Labels
apm.setLabel('order_id', orderId);
apm.setLabel('customer_tier', 'premium');

// User context
apm.setUserContext({
id: user.id,
username: user.name,
email: user.email,
});

transaction.end();

4 - SigNoz (Open Source)

4.1 Installation

# Docker
git clone https://github.com/SigNoz/signoz.git
cd signoz/deploy
./install.sh
# Kubernetes
helm repo add signoz https://charts.signoz.io
helm install signoz signoz/signoz \
--namespace monitoring \
--create-namespace

4.2 Application configuration

// Standard OpenTelemetry - compatible with SigNoz
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');

const sdk = new NodeSDK({
traceExporter: new OTLPTraceExporter({
url: 'http://signoz-otel-collector:4318/v1/traces',
}),
// ...
});

5 - Profiling

5.1 Continuous Profiling

5.2 Pyroscope

# docker-compose.yml
pyroscope:
image: pyroscope/pyroscope:latest
ports:
- "4040:4040"
command: ["server"]
// Node.js agent
const Pyroscope = require('@pyroscope/nodejs');

Pyroscope.init({
serverAddress: 'http://pyroscope:4040',
appName: 'my-service',
});

Pyroscope.start();

5.3 Flame Graphs

                                      [main]
┌─────────────────────┴──────────────────────┐
[processRequest] [handleResponse]
┌───────────┴───────────┐ │
[validateInput] [queryDatabase] [serialize]
│ │
[checkAuth] [executeSQL]

6 - Real User Monitoring (RUM)

6.1 Concept

6.2 Implementation

<!-- Elastic RUM -->
<script src="https://unpkg.com/@elastic/apm-rum"></script>
<script>
elasticApm.init({
serviceName: 'my-frontend',
serverUrl: 'https://apm.example.com',
environment: 'production',
});
</script>
// React
import { init as initApm } from '@elastic/apm-rum';

const apm = initApm({
serviceName: 'my-react-app',
serverUrl: 'https://apm.example.com',
});

// Manual transaction
const transaction = apm.startTransaction('user-checkout', 'user-interaction');
// ... checkout logic
transaction.end();

6.3 RUM metrics

MetricDescription
LCPLargest Contentful Paint
FIDFirst Input Delay
CLSCumulative Layout Shift
TTFBTime to First Byte
FCPFirst Contentful Paint

7 - APM Dashboards

7.1 Service Overview

panels:
- title: "Request Rate"
type: timeseries
query: sum(rate(http_requests_total[5m])) by (service)

- title: "Error Rate"
type: stat
query: |
sum(rate(http_requests_total{status=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m])) * 100
thresholds:
- color: green
value: 0
- color: red
value: 1

- title: "P99 Latency"
type: gauge
query: histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))

- title: "Top Endpoints"
type: table
query: topk(10, sum(rate(http_requests_total[5m])) by (endpoint))

7.2 Dependency Map


8 - APM Alerting

# alerts.yml
groups:
- name: apm-alerts
rules:
- alert: HighLatencyP99
expr: |
histogram_quantile(0.99,
sum(rate(http_request_duration_seconds_bucket[5m])) by (service, le)
) > 1
for: 5m
labels:
severity: warning
annotations:
summary: "High P99 latency on {{ $labels.service }}"

- alert: ApdexLow
expr: |
(
sum(rate(http_request_duration_seconds_bucket{le="0.5"}[5m])) by (service)
+ sum(rate(http_request_duration_seconds_bucket{le="2.0"}[5m])) by (service) / 2
)
/ sum(rate(http_request_duration_seconds_count[5m])) by (service)
< 0.8
for: 10m
labels:
severity: critical

Summary

In this chapter, we learned:

  • The APM concepts
  • The available solutions
  • Elastic APM and agents
  • SigNoz open source
  • Continuous Profiling
  • RUM (Real User Monitoring)
  • APM Dashboards and alerts

Next step

In the next chapter, we will look at Best practices.

→ Next chapter: Best practices


← Back to table of contents