Stage 2 · Tools
Network Scripting
curl — HTTP Automation
GET/POST/PUT/DELETE, headers, auth, follow redirects, and JSON payloads.
curl Fundamentals
curl transfers data to or from a server using various protocols. It is the universal HTTP client for shell scripts — essential for API integration, health checks, and web automation.
# Simple GET request
curl https://api.example.com/users
# Save output to file
curl -o file.zip https://example.com/download.zip
# Show response headers
curl -I https://api.example.com
# Verbose output (debugging)
curl -v https://api.example.com
# Silent mode (no progress bar)
curl -s https://api.example.com
# Follow redirects
curl -L https://example.comcurl outputs the response body to stdout by default. -o saves to a file. -I shows only headers. -v shows request/response details. -s suppresses progress. -L follows redirects.
In scripts, use curl -sS — silent mode suppresses the progress bar but still shows errors on stderr. This keeps output clean while maintaining error visibility.
HTTP Methods
curl supports all HTTP methods via -X. GET is the default. POST sends data. PUT updates. DELETE removes. PATCH partially updates.
# GET (default)
curl https://api.example.com/users
# POST with data
curl -X POST https://api.example.com/users \
-d "name=Alice&email=alice@example.com"
# PUT with JSON
curl -X PUT https://api.example.com/users/1 \
-H "Content-Type: application/json" \
-d '{"name":"Alice","email":"alice@new.com"}'
# DELETE
curl -X DELETE https://api.example.com/users/1
# PATCH
curl -X PATCH https://api.example.com/users/1 \
-H "Content-Type: application/json" \
-d '{"email":"updated@example.com"}'-X sets the HTTP method. -d sends form data (POST). For JSON, set Content-Type header and use -d with JSON string. -H sets custom headers.
Headers and Authentication
curl provides multiple ways to send headers and authentication credentials. Most APIs require an Authorization header.
# Custom headers
curl -H "Accept: application/json" \
-H "X-Custom-Header: value" \
https://api.example.com
# Basic auth
curl -u username:password https://api.example.com
# Bearer token
curl -H "Authorization: Bearer YOUR_TOKEN" \
https://api.example.com
# API key header
curl -H "X-API-Key: YOUR_KEY" https://api.example.com
# Multiple headers
curl -H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer TOKEN" \
https://api.example.com-H sets headers. -u sets basic auth (base64 encoded). For bearer tokens, set the Authorization header manually. Most modern APIs use bearer tokens or API keys.
JSON Requests
Working with JSON APIs is curl's most common use case. Combine -H, -d, and jq for complete API integration.
# POST JSON and parse response
curl -s -X POST https://api.example.com/users \
-H "Content-Type: application/json" \
-d '{"name":"Alice","role":"admin"}' | jq '.id'
# Read JSON from file
curl -X POST https://api.example.com/data \
-H "Content-Type: application/json" \
-d @payload.json
# Extract specific field from response
TOKEN=$(curl -s -X POST https://api.example.com/auth \
-H "Content-Type: application/json" \
-d '{"user":"admin","pass":"secret"}' | jq -r '.token')
# Use token in next request
curl -H "Authorization: Bearer $TOKEN" \
https://api.example.com/mejq -r extracts raw strings (no quotes). @payload.json reads JSON from a file. Chaining curl and jq lets you extract values from one API call and use them in another.
curl -sS -o /dev/null -w '%{http_code}' returns only the HTTP status code. Use this to check if requests succeeded: STATUS=$(curl -sS -o /dev/null -w '%{http_code}' URL).
File Downloads
curl handles file downloads with resume support, progress bars, and authentication. It is more flexible than wget for most download scenarios.
# Simple download
curl -O https://example.com/file.tar.gz
# Resume interrupted download
curl -C - -O https://example.com/large.iso
# Download with progress
curl -# -O https://example.com/file.tar.gz
# Follow redirects and download
curl -L -O https://github.com/owner/repo/archive/main.zip
# Limit download speed
curl --limit-rate 1M -O https://example.com/large.iso
# Download with auth
curl -u user:pass -O https://private.example.com/file.zip
# Check if URL exists without downloading
curl -sS -o /dev/null -w '%{http_code}' https://example.com/file-C - resumes from where the download left off. -L follows redirects (essential for GitHub). --limit-rate prevents bandwidth saturation. -# shows a progress bar.
Advanced Patterns
# Retry on failure
curl --retry 3 --retry-delay 5 https://api.example.com
# Timeout settings
curl --connect-timeout 5 --max-time 30 https://api.example.com
# Cookie handling
curl -c cookies.txt -b cookies.txt https://login.example.com
# Proxy support
curl -x http://proxy:8080 https://api.example.com
# Certificate pinning
curl --cacert /path/to/ca.crt https://secure.example.com
# Output formatted response time
curl -sS -o /dev/null -w "HTTP %{http_code} in %{time_total}s\n" URL--retry handles transient failures. --connect-timeout prevents hanging on unreachable hosts. -c/-b handle cookies for authenticated sessions. -w formats output with variables.
Never put API keys or passwords directly in scripts. Use environment variables: curl -H "Authorization: Bearer $API_KEY". Or use a credentials file with restricted permissions.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.