falcosecurity/falcosidekick repository overview

⁠Falcosidekick

⁠Description

A simple daemon to help you with falco's outputs (https://sysdig.com/opensource/falco/⁠). It takes a falco's event and forwards it to different outputs.

⁠Outputs

Currently available outputs are :

• Slack⁠
• Teams⁠
• Datadog⁠
• AlertManager⁠
• Elasticsearch⁠
• Loki⁠
• NATS⁠
• Influxdb⁠
• AWS Lambda⁠
• AWS SQS⁠
• SMTP (email)
• Opsgenie⁠
• StatsD⁠ (for monitoring of falcosidekick)
• DogStatsD⁠ (for monitoring of falcosidekick)
• Webhook

⁠Usage

Run the daemon as any other daemon in your architecture (systemd, k8s daemonset, swarm service, ...)

⁠With docker

docker run -d -p 2801:2801 -e SLACK_WEBHOOKURL=XXXX -e DATADOG_APIKEY=XXXX falcosecurity/falcosidekick

⁠With Helm

git clone https://github.com/falcosecurity/falcosidekick.git
cd ./falcosidekick/deploy/helm/falcosidekick/
helm install --name falcosidekick .

⁠Falco's config

Add this (adapted to your environment) in your falco.yaml :

json_output: true
json_include_output_property: true
http_output:
  enabled: true
  url: http://localhost:2801/"

or

json_output: true
json_include_output_property: true
program_output:
  enabled: true
  keep_alive: false
  program: "curl -d @- localhost:2801/"

⁠Configuration

Configuration is made by file (yaml) and env vars, both can be used but env vars override values from file.

⁠YAML File

See config_example.yaml :

#listenport: 2801 # port to listen for daemon (default: 2801)
debug: false # if true all outputs will print in stdout the payload they send (default: false)
customfields: # custom fields are added to falco events
  Akey: "AValue"
  Bkey: "BValue"
  Ckey: "CValue"

slack:
  webhookurl: "" # Slack WebhookURL (ex: https://hooks.slack.com/services/XXXX/YYYY/ZZZZ), if not empty, Slack output is enabled
  #footer: "" # Slack footer
  #icon: "" # Slack icon (avatar)
  outputformat: "text" # all (default), text, fields
  minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
  messageformat: "Alert : rule *{{ .Rule }}* triggered by user *{{ index .OutputFields \"user.name\" }}*" # a Go template to format Slack Text above Attachment, displayed in addition to the output from `SLACK_OUTPUTFORMAT`, see [Slack Message Formatting](#slack-message-formatting) in the README for details. If empty, no Text is displayed before Attachment.

teams:
  webhookurl: "" # Teams WebhookURL (ex: https://hooks.slack.com/services/XXXX/YYYY/ZZZZ), if not empty, Teams output is enabled
  #activityimage: "" # Image for message section
  outputformat: "text" # all (default), text, facts
  minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

datadog:
  #apikey: ""  # Datadog API Key, if not empty, Datadog output is enabled
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

alertmanager:
  # hostport: "" # http://{domain or ip}:{port}, if not empty, Alertmanager output is enabled
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

elasticsearch:
  # hostport: "" # http://{domain or ip}:{port}, if not empty, Elasticsearch output is enabled
  # index: "falco" # index (default: falco)
  # type: "event"
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
  # suffix: "daily" # date suffix for index rotation : daily (default), monthly, annually, none

influxdb:
  # hostport: "" # http://{domain or ip}:{port}, if not empty, Influxdb output is enabled
  # database: "falco" # Influxdb database (default: falco)
  # user: "" # user to use if auth is enabled in Influxdb
  # password: "" # pasword to use if auth is enabled in Influxdb
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

loki:
  # hostport: "" # http://{domain or ip}:{port}, if not empty, Loki output is enabled
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

nats:
  # hostport: "" # nats://{domain or ip}:{port}, if not empty, NATS output is enabled
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

aws:
  # accesskeyid: "" # aws access key (optionnal if you use EC2 Instance Profile)
  # secretaccesskey: "" # aws secret access key (optionnal if you use EC2 Instance Profile)
  # region : "" # aws region (optionnal if you use EC2 Instance Profile)
  lambda:
    # functionname : "" # Lambda function name, if not empty, AWS Lambda output is enabled
    # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
  sqs:
    # url : "" # SQS Queue URL, if not empty, AWS SQS output is enabled
    # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

smtp:
  # hostport: "" # host:port address of SMTP server, if not empty, SMTP output is enabled
  # user: "" # user to access SMTP server
  # password: "" # password to access SMTP server
  # from: "" # Sender address (mandatory if SMTP output is enabled)
  # to: "" # comma-separated list of Recipident addresses, can't be empty (mandatory if SMTP output is enabled)
  # outputformat: "" # html (default), text
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

opsgenie:
  # apikey: "" # Opsgenie API Key, if not empty, Opsgenie output is enabled
  # region: "eu" # (us|eu) region of your domain (default is 'us')
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

statsd:
  forwarder: "" # The address for the StatsD forwarder, in the form "host:port", if not empty StatsD is enabled
  namespace: "falcosidekick." # A prefix for all metrics (default: "falcosidekick.")

dogstatsd:
  forwarder: "" # The address for the DogStatsD forwarder, in the form "host:port", if not empty DogStatsD is enabled
  namespace: "falcosidekick." # A prefix for all metrics (default: "falcosidekick.")
  # tag :
  #   key: "value"

webhook:
  # address: "" # Webhook address, if not empty, Webhook output is enabled
  # minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

Usage :

usage: falcosidekick [<flags>]

Flags:
      --help                     Show context-sensitive help (also try --help-long and --help-man).
  -c, --config-file=CONFIG-FILE  config file

⁠Env vars

Configuration of the daemon can be made also by env vars, these values override these from yaml file.

The env vars "match" field names in *yaml file with this structure (take care of lower/uppercases) : yaml: a.b --> envvar: A_B :

• LISTENPORT : port to listen for daemon (default: 2801)
• DEBUG : if true all outputs will print in stdout the payload they send (default: false)
• CUSTOMFIELDS : a list of comma separated custom fields to add to falco events, syntax is "key:value,key:value"
• SLACK_WEBHOOKURL : Slack Webhook URL (ex: https://hooks.slack.com/services/XXXX/YYYY/ZZZZ⁠), if not empty, Slack output is enabled
• SLACK_FOOTER : Slack footer
• SLACK_ICON : Slack icon (avatar)
• SLACK_OUTPUTFORMAT : all (default), text (only text is displayed in Slack), fields (only fields are displayed in Slack)
• SLACK_MINIMUMPRIORITY : minimum priority of event for using use this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• SLACK_MESSAGEFORMAT : a Go template to format Slack Text above Attachment, displayed in addition to the output from SLACK_OUTPUTFORMAT, see Slack Message Formatting⁠ in the README for details. If empty, no Text is displayed before Attachment.
• TEAMS_WEBHOOKURL : Teams Webhook URL (ex: https://outlook.office.com/webhook/XXXXXX/IncomingWebhook/YYYYYY⁠"), if not empty, Teams output is enabled
• TEAMS_ACTIVITYIMAGE : Teams section image
• TEAMS_OUTPUTFORMAT : all (default), text (only text is displayed in Teams), facts (only facts are displayed in Teams)
• TEAMS_MINIMUMPRIORITY : minimum priority of event for using use this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• DATADOG_APIKEY : Datadog API Key, if not empty, Datadog output is enabled
• DATADOG_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• ALERTMANAGER_HOSTPORT : AlertManager http://host:port⁠, if not empty, AlertManager is enabled
• ALERTMANAGER_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• ELASTICSEARCH_HOSTPORT : Elasticsearch http://host:port⁠, if not empty, Elasticsearch is enabled
• ELASTICSEARCH_INDEX : Elasticsearch index (default: falco)
• ELASTICSEARCH_TYPE : Elasticsearch document type (default: event)
• ELASTICSEARCH_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• ELASTICSEARCH_SUFFIX : date suffix for index rotation : daily (default), monthly, annually, none
• INFLUXDB_HOSTPORT : Influxdb http://host:port⁠, if not empty, Influxdb is enabled
• INFLUXDB_DATABASE : Influxdb database (default: falco)
• INFLUXDB_USER : user to use if auth is enabled in Influxdb
• INFLUXDB_PASSWORD : user to use if auth is enabled in Influxdb
• INFLUXDB_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• LOKI_HOSTPORT : Loki http://host:port⁠, if not empty, Loki is enabled
• LOKI_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• NATS_HOSTPORT : NATS "nats://host:port", if not empty, NATS is enabled
• NATS_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• AWS_ACCESSKEYID : AWS Access Key Id (optionnal if you use EC2 Instance Profile)
• AWS_SECRETACCESSKEY : AWS Secret Access Key (optionnal if you use EC2 Instance Profile)
• AWS_REGION : AWS Region (optionnal if you use EC2 Instance Profile)
• AWS_LAMBDA_FUNCTIONNAME : AWS Lambda Function Name, if not empty, AWS Lambda output is enabled
• AWS_LAMBDA_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• AWS_SQS_URL : AWS SQS Queue URL, if not empty, AWS SQS output is enabled
• AWS_SQS_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• SMTP_HOSTPORT : "host:port" address of SMTP server, if not empty, SMTP output is enabled
• SMTP_USER : user to access SMTP server
• SMTP_PASSWORD : password to access SMTP server
• SMTP_FROM : Sender address (mandatory if SMTP output is enabled)
• SMTP_TO : comma-separated list of Recipident addresses, can't be empty (mandatory if SMTP output is enabled)
• SMTP_OUTPUTFORMAT : "" # html (default), text
• SMTP_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• OPSGENIE_APIKEY : Opsgenie API Key, if not empty, Opsgenie output is enabled
• OPSGENIE_REGION : "" # (us|eu) region of your domain (default is 'us')
• OPSGENIE_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
• STATSD_FORWARDER: The address for the StatsD forwarder, in the form http://host:port⁠, if not empty StatsD is enabled
• STATSD_NAMESPACE: A prefix for all metrics (default: "falcosidekick.")
• DOGSTATSD_FORWARDER: The address for the DogStatsD forwarder, in the form http://host:port⁠, if not empty DogStatsD is enabled
• DOGSTATSD_NAMESPACE: A prefix for all metrics (default: falcosidekick."")
• DOGSTATSD_TAGS: A comma-separated list of tags to add to all metrics
• WEBHOOK_ADDRESS : "" # Webhook address, if not empty, Webhook output is enabled
• WEBHOOK_MINIMUMPRIORITY : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

⁠Slack Message Formatting

The SLACK_MESSAGEFORMAT environment variable and slack.messageformat YAML value accept a Go template⁠ which can be used to format the text of a slack alert. These templates are evaluated on the JSON data from each Falco event - the following fields are available:

Go templates also support some basic methods for text manipulation which can be used to improve the clarity of alerts - see the documentation for details.

⁠Handlers

Different URI (handlers) are available :

• / : main and default handler, your falco config must be configured to use it
• /ping : you will get a pong as answer, useful to test if falcosidekick is running and its port is opened (for healthcheck purpose for example)
• /test : (for debug only) send a test event to all enabled outputs.
• /debug/vars : get statistics from daemon (in JSON format), it uses classic expvar package and some custom values are added

⁠Logs

All logs are sent to stdout.

2019/05/10 14:32:06 [INFO] : Enabled Outputs : Slack Datadog

⁠Metrics

⁠Golang ExpVar

The daemon exposes the common Golang metrics and some custom values in JSON format. It's useful for monitoring purpose.

⁠StatsD

The daemon is able to push its metrics to a StatsD server. See Configuration⁠ section for how-to.

⁠Examples

Run you daemon and try (from falco's documentation) :

curl "http://localhost:2801/" -d'{"output":"16:31:56.746609046: Error File below a known binary directory opened for writing (user=root command=touch /bin/hack file=/bin/hack)","priority":"Error","rule":"Write below binary dir","time":"2019-05-17T15:31:56.746609046Z", "output_fields": {"evt.time":1507591916746609046,"fd.name":"/bin/hack","proc.cmdline":"touch /bin/hack","user.name":"root"}}'

You should get :

⁠Slack

(SLACK_OUTPUTFORMAT="all") (SLACK_OUTPUTFORMAT="text") (SLACK_OUTPUTFORMAT="fields" and SLACK_MESSAGEFORMAT="Alert : rule *{{ .Rule }}* triggered by user *{{ index .OutputFields "user.name" }}*")

⁠Teams

(TEAMS_OUTPUTFORMAT="all") (TEAMS_OUTPUTFORMAT="text")

⁠Datadog

(Tip: filter on sources: falco)

⁠AlertManager

⁠Elasticsearch (with Kibana)

⁠Influxdb

> use falco
Using database falco
> show series
key
---
events,akey=AValue,bkey=BValue,ckey=CValue,priority=Debug,rule=Testrule
events,akey=A_Value,bkey=B_Value,ckey=C_Value,priority=Debug,rule=Test_rule
> select * from events
name: events
time                akey    bkey    ckey    priority rule      value
----                ----    ----    ----    -------- ----      -----
1560433816893368400 AValue  BValue  CValue  Debug    Testrule  This is a test from falcosidekick
1560441359119741800 A_Value B_Value C_Value Debug    Test_rule This is a test from falcosidekick

⁠Loki (with Grafana)

⁠AWS SQS

⁠SMTP

(SMTP_OUTPUTFORMAT="html") (SMTP_OUTPUTFORMAT="text")

⁠Opsgenie

⁠Development

⁠Build

go build

⁠Test & Coverage

go test ./outputs -count=1 -cover -v

⁠Author

Thomas Labarussias (https://github.com/Issif⁠)