Skip to main content
Sumo Logic

Collect Logs for Fastly

Set up Fastly as an HTTP Source and create a Sumo Logic endpoint within Fastly to begin sending Fastly data to Sumo Logic.

Collect Logs for Sumo Logic Fastly App

To collect logs for the Sumo Logic Fastly App, perform the following steps, detailed in the sections below:

  1. Add a Sumo Logic Hosted Collector to your Sumo Logic Org

  2. Configure an HTTP Source for your Fastly data. Make sure to set the Source Category when configuring the HTTP source. For example, fastly.

  3. Configure Fastly to send Sumo Logic the data in Log format version to 2 . (You can use the Fastly Documentation to determine which log file format you are using.)

    Fastly Create a Sumo Logic Endpoint

    After Step 5 in the Fastly documentation, you must set the log line format to "blank" for each configuration:
    FastlyConfigureLogtoBlank

And then use this format string which generates the necessary JSON output:


And then use this format string which generates the necessary JSON output:
 
{
 "service_id":"%{req.service_id}V",
 "service_version":"%{fastly_info.version}V",
 "time_start":"%{begin:Y-%m-%dT%H:%M:%S%Z}t",
 "time_end":"%{end:%Y-%m-%dT%H:%M:%S%Z}t",
 "time_elapsed":%{time.elapsed.usec}V,
 "client_ip":"%{req.http.Fastly-Client-IP}V",
 "request":"%{req.request}V",
 "protocol":"%{req.proto}V",
 "host":"%{req.http.Fastly-Orig-Host}V",
 "origin_host":"%{req.http.Host}V",
 "url":"%{cstr_escape(req.url)}V",
 "is_ipv6":%{if(req.is_ipv6, "true", "false")}V,
 "is_tls":%{if(req.is_ssl, "true", "false")}V,
 "tls_client_protocol":"%{cstr_escape(tls.client.protocol)}V",
 "tls_client_servername":"%{cstr_escape(tls.client.servername)}V",
 "tls_client_cipher":"%{cstr_escape(tls.client.cipher)}V",
 "tls_client_cipher_sha":"%{cstr_escape(tls.client.ciphers_sha )}V",
 "tls_client_tlsexts_sha":"%{cstr_escape(tls.client.tlsexts_sha)}V",
 "is_h2":%{if(fastly_info.is_h2, "true", "false")}V,
 "is_h2_push":%{if(fastly_info.h2.is_push, "true", "false")}V,
 "h2_stream_id":"%{fastly_info.h2.stream_id}V",
 "request_referer":"%{cstr_escape(req.http.Referer)}V",
 "request_user_agent":"%{cstr_escape(req.http.User-Agent)}V",
 "request_accept_content":"%{cstr_escape(req.http.Accept)}V",
 "request_accept_language":"%{cstr_escape(req.http.Accept-Language)}V",
 "request_accept_encoding":"%{cstr_escape(req.http.Accept-Encoding)}V",
 "request_accept_charset":"%{cstr_escape(req.http.Accept-Charset)}V",
 "request_connection":"%{cstr_escape(req.http.Connection)}V",
 "request_dnt":"%{cstr_escape(req.http.DNT)}V",
 "request_forwarded":"%{cstr_escape(req.http.Forwarded)}V",
 "request_via":"%{cstr_escape(req.http.Via)}V",
 "request_cache_control":"%{cstr_escape(req.http.Cache-Control)}V",
 "request_x_requested_with":"%{cstr_escape(req.http.X-Requested-With)}V",
 "request_x_forwarded_for":"%{cstr_escape(req.http.X-Forwarded-For)}V",
 "status":"%{resp.status}V",
 "content_type":"%{cstr_escape(resp.http.Content-Type)}V",
 "cache_status":"%{regsub(fastly_info.state, "^(HIT-(SYNTH)|(HITPASS|HIT|MISS|PASS|ERROR|PIPE)).*", "\\2\\3")}V",
 "is_cacheable":%{if(fastly_info.state ~"^(HIT|MISS)$", "true", "false")}V,
 "response_age":"%{cstr_escape(resp.http.Age)}V",
 "response_cache_control":"%{cstr_escape(resp.http.Cache-Control)}V",
 "response_expires":"%{cstr_escape(resp.http.Expires)}V",
 "response_last_modified":"%{cstr_escape(resp.http.Last-Modified)}V",
 "response_tsv":"%{cstr_escape(resp.http.TSV)}V",
 "geo_datacenter":"%{server.datacenter}V",
 "geo_city":"%{geoip.city}V",
 "geo_country_code":"%{geoip.country_code}V",
 "geo_continent_code":"%{geoip.continent_code}V",
 "geo_region":"%{geoip.region}V",
 "req_header_size":%{req.header_bytes_read}V,
 "req_body_size":%{req.body_bytes_read}V,
 "resp_header_size":%{resp.header_bytes_written}V,
 "resp_body_size":%{resp.body_bytes_written}V,
 "socket_cwnd":%{client.socket.cwnd}V,
 "socket_nexthop":"%{client.socket.nexthop}V",
 "socket_tcpi_rcv_mss":%{client.socket.tcpi_rcv_mss}V,
 "socket_tcpi_snd_mss":%{client.socket.tcpi_snd_mss}V,
 "socket_tcpi_rtt":%{client.socket.tcpi_rtt}V,
 "socket_tcpi_rttvar":%{client.socket.tcpi_rttvar}V,
 "socket_tcpi_rcv_rtt":%{client.socket.tcpi_rcv_rtt}V,
 "socket_tcpi_rcv_space":%{client.socket.tcpi_rcv_space}V,
 "socket_tcpi_last_data_sent":%{client.socket.tcpi_last_data_sent}V,
 "socket_tcpi_total_retrans":%{client.socket.tcpi_total_retrans}V,
 "socket_tcpi_delta_retrans":%{client.socket.tcpi_delta_retrans}V,
 "socket_ploss":%{client.socket.ploss}V
}

Once you've done that hit save, deploy your version and your Sumo Logic account will start populating with data.

Field Extraction Rule

This Field Extraction Rule (FER) is provided as an example to help you reduce your overall parsing time. To create this FER see Creating a Field Extraction Rule.

parse "\"reqMethod\":\"*\"" as method, "\"status\":\"*\"" as status, "\"fwdHost\":\"*\"" as origin| parse "\"bytes\":\"*\"" as bytes, "\"edgeIP\":\"*\"" as edgeip, "\"country\":\"*\"" as country, "\"cookie\":\"*\"" as cookie

Sample Queries

Top Error-causing URLs

_sourceCategory=fastly 50?
| parse "\"reqPath\":\"*\"" as path, "\"status\":\"*\"" as status
| urldecode(path) as path
| where status > 499
| where status < 600
| count as errors by path
| sort by errors

Cache Performance

_sourceCategory=fastly cacheStatus
| parse "\"cacheStatus\":\"*\"" as status
| where !(status="")
| if(status="0", "0 - Non cacheable", if(status="1" OR status="2", "1/2 - Cache Hit", if(status="3", "3 - Cache Miss", ""))) as cachestatus
| count by cachestatus

Top Denials by Host

_sourceCategory=fastly waf denyRules reqHost
| parse "\"denyRules\":\"*\"" as deny, "\"reqHost\":\"*\"" as host
| where deny != ""
| timeslice 1m
| count by host, _timeslice
| transpose row _timeslice column host