It is a paid option
You can receive CDN resource logs from CDN servers and shielding (if it is activated).
Contact us via chat or email to support@gcorelabs.com to enable the option.
Once the feature is enabled you will be able to set it up in your personal account->CDN section->Raw logs->Receive raw logs.
We use two protocols to deliver logs: S3 and FTP.
You should set up a storage on your side to start receiving the logs.
Configure logs receiving on S3
Configure logs receiving on FTP
Configure logs receiving on S3
Storage is Amazon
If you want to receive logs to your Amazon storage specify AWS Access Key ID, AWS Secret Access Key and Region (optional).
Check the box Add logs from origin shielding to receive logs from CDN servers and shielding.
Select where you want to send logs:
- into one folder
- into different folders
Note! The system doesn't create folders (bucket) in your storage. Specify an existing folder (bucket).
Click Save Changes.
Storage is not Amazon
If you choose Other, fill in Hostname, Access Key ID and Secret Access Key.
If you use Yandex.Cloud or our S3 storage, fill in Bucket hostname.
Check the box Add logs from origin shielding to receive logs from CDN servers and shielding.
Select where you want to send logs:
- into one folder
- into different folders
Note! The system doesn't create folders (bucket) in your storage. Specify an existing folder (bucket).
Click Save Changes.
Configure logs receiving on FTP
Specify Hostname, Login, Password and Prepend folder (optional).
Note! A port number can be added to your Hostname if needed: <hostname>:<port>.
Check the box Add logs from origin shielding to receive logs from CDN servers and shielding.
Select where you want to send logs:
- into one folder
- into different folders
Note! The system doesn't create folders (bucket) in your storage. Specify an existing folder (bucket).
Click Save Changes.
Log path example
s3://log-bucket-name/2019/08/20/15/nodename_primarycname.gcdn.co_access.log.gz
Log Format
"$remote_addr" "-" "$remote_user" "[$time_local]" "$request" "$status"
"$body_bytes_sent" "$http_referer" "$http_user_agent" "$bytes_sent"
"$sent_http_content_size" "$scheme" "$host" "$request_time"
"$upstream_response_time" "$request_length" "$http_range" "[$responding_node]"
"$upstream_cache_status" "$upstream_response_length" "$upstream_addr"
"$gcdn_api_client_id" "$gcdn_api_resource_id" "$uid_got" "$uid_set"
"$geoip_city_country_code" "$geoip_city" "$shield_type" "$server_addr" "$server_port"
"$upstream_status" "-" "$upstream_connect_time" "$upstream_header_time"
"$shard_addr" "$geoip2_data_asnumber" "$connection" "$connection_requests"
"$request_id" "$http_x_forwarded_proto" "$http_x_forwarded_request_id" "$ssl_cipher"
"$ssl_session_id" "$ssl_session_reused";
The log format can be changed at any time. New fields are always added to the end of the line. When analyzing logs, only the known fields from the beginning of the line should be taken into account.
Example Log
"0.0.0.0" "-" "-" "[26/Apr/2019:09:47:40 +0000]" "GET /ContentCommon/images/image.png HTTP/1.1" "200" "1514283" "https://example.com/videos/10" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 YaBrowser/16.10.0.2309 Safari/537.36" "1514848" "-" "https" "origin.cdn.com" "1.500" "0.445" "157" "-" "[kal]" "MISS" "10485760" "0.0.0.0:80" "2510" "7399" "-" "-" "KZ" "-" "shield_no" "0.0.0.0" "80" "206" "-" "0.000" "0.200" "0.0.0.0" "asnumber" "106980391" "1" "c1c0f12ab35b7cccccd5dc0a454879c5" "-" "-" "ECDHE-RSA-AES256-GCM-SHA384" "28a4184139cb43cdc79006cf2d1a4ac93bdc****" "r"
Log Fields
| Field | Example log value |
Description |
| $remote_addr | 0.0.0.0 |
IP address of the client |
| $remote_user | - |
User name supplied with the Basic authentication |
| [$time_local] | [26/Apr/2019:09:47:40 +0000] |
Local time in the Common Log Format |
| $request |
GET /ContentCommon/images/image.png HTTP/1.1 |
HTTP request type, requested path to a file, HTTP version |
| $status |
200 |
Response status |
| $body_bytes_sent |
1514283 |
Number of bytes sent to a client, disregarding the response header |
| $http_referer |
https://example.com/videos/10 |
Referrer: specify the URL of the user |
| $http_user_agent |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 YaBrowser/16.10.0.2309 Safari/537.36 |
User Agent which a user applied to request content |
| $bytes_sent |
1514848 |
Number of bytes sent to a client |
| $sent_http_content_size | - |
Value of the HTTP Content-Size header |
| $scheme | https |
Request's protocol (HTTP ot HTTPS) |
| $host | cdn.example.com |
CDN-resource's Hostname received from a request |
| $request_time | 1.500 |
Request processing time counted in seconds with a milliseconds resolution; time elapsed from the first bytes sent by the client till the log record, made after a client sent last bytes |
| $upstream_response_time | 0.445 |
Time spent on receiving the response from the upstream server; time counted in seconds with millisecond resolution. Time of several responses is shown in commas and colon |
| $request_length | 157 |
Request length (including request line, header and request body) |
| $http_range | - |
File fragment value received by Range request |
| [$responding_node] | kal |
The name of the responded CDN-node |
| $upstream_cache_status | MISS |
CDN cache status of a requested file: HIT – file's given from the cache STALE – file outdated as the origin server didn't respond or responded incorrectly due to cache's being updated UPDATING - file outdated as it's being updated after the earlier request was sent REVALIDATED – the proxy_cache_revalidate directive was enabled and NGINX verified that the currently cached content is still valid EXPIRED - cache lifetime ended, but the file is relevant to the origin's file. A request is sent to the origin server for the recache MISS – file isn't given from the cache, it's proxied from the origin server |
| $upstream_response_length | 10485760 |
Length of the response obtained from the origin server; stored in bytes. Lengths of several responses are separated by commas and colons |
| $upstream_addr | 0.0.0.0:80 |
IP address and port of the prigin server |
| $gcdn_api_client_id | 123 |
Your ID in our system |
| $gcdn_api_resource_id | 01 |
Your CDN-resource ID in our system |
| $uid_got | - |
Cookie name and received user ID |
| $uid_set | - |
Cookie name and provided user ID |
| $geoip_country_code | KZ |
User’s country code |
| $geoip_city | - |
User’s city code |
| $shield_type | shield_no |
Specifies if a shield is enabled or not shield_old - enabled shield_no - disabled
|
| $server_addr | 0.0.0.0 |
IP address of anycast zone or edge-server |
| $server_port | 80 |
Port of the server which accepted a request |
| $upstream_status | 206 |
Origin server's response code |
| $upstream_connect_time | 0.000 |
Time spent to access the origin server; time counted in seconds with millisecond resolution. |
| $upstream_header_time | 0.200 |
Time spent to receive a header from the origin server; time counted in seconds with millisecond resolution. |
| $shard_addr | 0.0.0.0 |
IP address of an edge-server that received the first client request if the Cache Sharding feature is enabled. |
| $geoip2_data_asnumber | asnumber |
The autonomous system number where the user request came from. |
| $connection_requests | 1 |
Current number of requests made through a connection. |
| $request_id | c1c0f12ab35b7cccccd5dc0a454879c5 |
Unique request identifier generated from 16 random bytes, in hexadecimal. |
| $http_x_forwarded_proto | - |
Initial protocol of the incoming request (http or https). |
| $http_x_forwarded_request_id | - |
Initial ID of the incoming request. |
| $ssl_cipher | ECDHE-RSA-AES256-GCM-SHA384 |
Returns the name of the cipher used for an established SSL connection. |
| $ssl_session_id | 28a4184139cb43cdc79006cf2d1a4ac93bdc**** |
Returns the session identifier of an established SSL connection. |
| $ssl_session_reused | r |
Returns “ |