CloudWatch Logs

Natively integrated with most AWS services.

Some services automatically create logs for you when you use them.

Eg. EC2, lambda, ECS

CloudTrail

Records AWS API calls - who used what service to do what.

Think of it as git blame for AWS.

• Example Log

  {
    "eventVersion": "1.10",
    "userIdentity": {
      "type": "Root",
      "principalId": "263311111159",
      "arn": "arn:aws:iam::263311111159:root",
      "accountId": "263311111159",
      "accessKeyId": "ASIAT3FYDDDDDBG6HDJM",
      "userName": "hahahaha",
      "sessionContext": {
        "attributes": {
          "creationDate": "2024-10-21T11:37:35Z",
          "mfaAuthenticated": "true"
        }
      }
    },
    "eventTime": "2024-10-21T11:39:08Z",
    "eventSource": "cloudtrail.amazonaws.com",
    "eventName": "DeleteTrail",
    "awsRegion": "ap-northeast-1",
    "sourceIPAddress": "MY IP ADDRESS",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:131.0) Gecko/20100101 Firefox/131.0",
    "requestParameters": {
      "name": "arn:aws:cloudtrail:ap-northeast-1:263311111159:trail/emflesfs"
    },
    "responseElements": null,
    "requestID": "67cd8160-6116-48b1-9334-7623b122f7e1",
    "eventID": "0786ec8d-83bf-4a56-a23b-398990f5ade9",
    "readOnly": false,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "263311111159",
    "eventCategory": "Management",
    "tlsDetails": {
      "tlsVersion": "TLSv1.3",
      "cipherSuite": "TLS_AES_128_GCM_SHA256",
      "clientProvidedHostHeader": "cloudtrail.ap-northeast-1.amazonaws.com"
    },
    "sessionCredentialFromConsole": "true"
  },

X-Ray

Think of it as the visual version of CloudTrail.

Helps you understand the flow direction, relationships, and latency between services.

EventBridge

Also known as CloudWatch Events.

Acts as a bridge between AWS services.

For example:

• CloudWatch Logs -> EventBridge -> trigger functions, send text or email, or to SQS queue.

• Or receive from other services and store in CloudWatch Logs:
  • EventBridge → Lambda → CloudWatch Logs

VPC Flow Logs

Can be created under VPC features > Action > Create flow logs

You can choose to monitor

• VPC level (entire VPC traffic)

• Subnet level (specific subnet only).

• ENI

• Transit Gateway

• Site-to-Site VPN.

The output is ultimately sent to CloudWatch Logs.

Amazon OpenSearch (formerly ElasticSearch)

Uses Kibana for visualization.

• Great for search and real-time log analysis, data processing

• Particularly good at handling unstructured data (compared to QuickSight, which is typically used for structured data)

• OpenSearch is more focused on existing data analysis, while X-Ray tracks service flows

Combinations

Just Google Image search "tool a tool b aws" and you'll find many architecture diagrams.

1. simple: CloudWatch + CloudTrail

2. deeper: CloudWatch + X-Ray

3. professional: CloudWatch → Kinesis → OpenSearch