// Use these for links to issue and pulls. Note issues and pulls redirect one to
// each other on Github, so don't worry too much on using the right prefix.
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

=== Beats version HEAD
https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD diff]

==== Breaking changes

*Affecting all Beats*

- Update to Golang 1.12.1. {pull}11330[11330]
- Disable Alibaba Cloud and Tencent Cloud metadata providers by default. {pull}13812[12812]
- Libbeat: Do not overwrite agent.*, ecs.version, and host.name. {pull}14407[14407]
- Libbeat: Cleanup the x-pack licenser code to use the new license endpoint and the new format. {pull}15091[15091]
- Refactor metadata generator to support adding metadata across resources {pull}14875[14875]
- Variable substitution from environment variables is not longer supported. {pull}15937{15937}
- Change aws_elb autodiscover provider field name from elb_listener.* to aws.elb.*. {issue}16219[16219] {pull}16402{16402}
- Remove `AddDockerMetadata` and `AddKubernetesMetadata` processors from the `script` processor. They can still be used as normal processors in the configuration. {issue}16349[16349] {pull}16514[16514]
- Introduce APM libbeat instrumentation, active when running the beat with ELASTIC_APM_ACTIVE=true. {pull}17938[17938]
- Make error message about locked data path actionable. {pull}18667[18667]

*Auditbeat*


*Filebeat*

- Fix parsing of Elasticsearch node name by `elasticsearch/slowlog` fileset. {pull}14547[14547]
- Improve ECS field mappings in panw module.  event.outcome now only contains success/failure per ECS specification. {issue}16025[16025] {pull}17910[17910]
- Improve ECS categorization field mappings for nginx module. http.request.referrer only populated when nginx sets a value {issue}16174[16174] {pull}17844[17844]
- Improve ECS field mappings in santa module. move hash.sha256 to process.hash.sha256 & move certificate fields to santa.certificate . {issue}16180[16180] {pull}17982[17982]
- With the default configuration the cloud modules (aws, azure, googlecloud, o365, okta)
`forwarded` from the list. {issue}13920[13920]
* Cisco {pull}18753[18753]
* CrowdStrike {pull}19132[19132]
* Fortinet {pull}19133[19133]
* iptables {pull}18756[18756]
* Checkpoint {pull}18754[18754]
* Netflow {pull}19087[19087]
* Zeek {pull}19113[19113] (`forwarded` tag is not included by default)
* Suricata {pull}19107[19107] (`forwarded` tag is not included by default)
* CoreDNS {pull}19134[19134] (`forwarded` tag is not included by default)
* Envoy Proxy {pull}19134[19134] (`forwarded` tag is not included by default)
- Preserve case of http.request.method.  ECS prior to 1.6 specified normalizing to lowercase, which lost information. Affects filesets: apache/access, elasticsearch/audit, iis/access, iis/error, nginx/access, nginx/ingress_controller, aws/elb, suricata/eve, zeek/http. {issue}18154[18154] {pull}18359[18359]
- With the default configuration the cloud modules (aws, azure, googlecloud, o365, okta)
- With the default configuration the cef and panw modules will no longer send the `host`
`forwarded` from the list. {issue}13920[13920] {pull}18223[18223]
- Adds oauth support for httpjson input. {issue}18415[18415] {pull}18892[18892]
- Adds `split_events_by` option to httpjson input. {pull}19246[19246]
- Adds `date_cursor` option to httpjson input. {pull}19483[19483]
- Adds Gsuite module with SAML support. {pull}19329[19329]
- Adds Gsuite User Accounts support. {pull}19329[19329]
- Adds Gsuite Login audit support. {pull}19702[19702]
- Adds Gsuite Admin support. {pull}19769[19769]
- Adds Gsuite Drive support. {pull}19704[19704]
- Adds Gsuite Groups support. {pull}19725[19725]

*Heartbeat*


*Journalbeat*

- Improve parsing of syslog.pid in journalbeat to strip the username when present {pull}16116[16116]


*Metricbeat*

- Add new dashboard for VSphere host cluster and virtual machine {pull}14135[14135]
- kubernetes.container.cpu.limit.cores and kubernetes.container.cpu.requests.cores are now floats. {issue}11975[11975]
- Fix ECS compliance of user.id field in system/users  metricset {pull}19019[19019]

*Packetbeat*

- Added redact_headers configuration option, to allow HTTP request headers to be redacted whilst keeping the header field included in the beat. {pull}15353[15353]
- Add dns.question.subdomain and dns.question.top_level_domain fields. {pull}14578[14578]

*Winlogbeat*

- Add support to Sysmon file delete events (event ID 23). {issue}18094[18094]
- Improve ECS field mappings in Sysmon module. `related.hash`, `related.ip`, and `related.user` are now populated. {issue}18364[18364]
- Improve ECS field mappings in Sysmon module. Hashes are now also populated to the corresponding `process.hash`, `process.pe.imphash`, `file.hash`, or `file.pe.imphash`. {issue}18364[18364]
- Improve ECS field mappings in Sysmon module. `file.name`, `file.directory`, and `file.extension` are now populated. {issue}18364[18364]
- Improve ECS field mappings in Sysmon module. `rule.name` is populated for all events when present. {issue}18364[18364]
- Add Powershell module. Support for event ID's: `400`, `403`, `600`, `800`, `4103`, `4014`, `4105`, `4106`. {issue}16262[16262] {pull}18526[18526]
- Fix Powershell processing of downgraded engine events. {pull}18966[18966]
- Fix unprefixed fields in `fields.yml` for Powershell module {issue}18984[18984]

*Functionbeat*


==== Bugfixes

*Affecting all Beats*

- Fix a race condition with the Kafka pipeline client, it is possible that `Close()` get called before `Connect()` . {issue}11945[11945]
- Allow users to configure only `cluster_uuid` setting under `monitoring` namespace. {pull}14338[14338]
- Update replicaset group to apps/v1 {pull}15854[15802]
- Fix Kubernetes autodiscovery provider to correctly handle pod states and avoid missing event data {pull}17223[17223]
- Fix `add_cloud_metadata` to better support modifying sub-fields with other processors. {pull}13808[13808]
- Fix missing output in dockerlogbeat {pull}15719[15719]
- Do not load dashboards where not available. {pull}15802[15802]
- Fix issue where TLS settings would be ignored when a forward proxy was in use. {pull}15516{15516}
- Remove superfluous use of number_of_routing_shards setting from the default template. {pull}16038[16038]
- Fix index names for indexing not always guaranteed to be lower case. {pull}16081[16081]
- Upgrade go-ucfg to latest v0.8.1. {pull}15937{15937}
- Fix loading processors from annotation hints. {pull}16348[16348]
- Upgrade go-ucfg to latest v0.8.3. {pull}16450{16450}
- Add `ssl.ca_sha256` option to the supported TLS option, this allow to check that a specific certificate is used as part of the verified chain. {issue}15717[15717]
- Fix `NewContainerMetadataEnricher` to use default config for kubernetes module. {pull}16857[16857]
- Improve some logging messages for add_kubernetes_metadata processor {pull}16866{16866}
- Do not rotate log files on startup when interval is configured and rotateonstartup is disabled. {pull}17613[17613]
- Fix `setup.dashboards.index` setting not working. {pull}17749[17749]
- Fix goroutine leak and Elasticsearch output file descriptor leak when output reloading is in use. {issue}10491[10491] {pull}17381[17381]
- Fix Elasticsearch license endpoint URL referenced in error message. {issue}17880[17880] {pull}18030[18030]
- Change `decode_json_fields` processor, to merge parsed json objects with existing objects in the event instead of fully replacing them. {pull}17958[17958]
- Gives monitoring reporter hosts, if configured, total precedence over corresponding output hosts. {issue}17937[17937] {pull}17991[17991]
- [Autodiscover] Check if runner is already running before starting again. {pull}18564[18564]
- Fix `keystore add` hanging under Windows. {issue}18649[18649] {pull}18654[18654]
- Fix regression in `add_kubernetes_metadata`, so configured `indexers` and `matchers` are used if defaults are not disabled. {issue}18481[18481] {pull}18818[18818]
- Fix the `translate_sid` processor's handling of unconfigured target fields. {issue}18990[18990] {pull}18991[18991]
- Fixed a service restart failure under Windows. {issue}18914[18914] {pull}18916[18916]
- Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed {pull}18979[18979]
- Server-side TLS config now validates certificate and key are both specified {pull}19584[19584]
- Fix index template loading when the new index format is selected. {issue}22482[22482] {pull}22682[22682]

*Auditbeat*

- system/socket: Fixed compatibility issue with kernel 5.x. {pull}15771[15771]
- system/package: Fix parsing of Installed-Size field of DEB packages. {issue}16661[16661] {pull}17188[17188]
- system module: Fix panic during initialisation when /proc/stat can't be read. {pull}17569[17569]
- system/package: Fix an error that can occur while trying to persist package metadata. {issue}18536[18536] {pull}18887[18887]
- system/socket: Fix dataset using 100% CPU and becoming unresponsive in some scenarios. {pull}19033[19033] {pull}19764[19764]
- system/socket: Fixed tracking of long-running connections. {pull}19033[19033]
- auditd: Fix an error condition causing a lot of `audit_send_reply` kernel threads being created. {pull}22673[22673]
- system/socket: Fixed start failure when run under config reloader. {issue}20851[20851] {pull}21693[21693]

*Filebeat*

- cisco/asa fileset: Fix parsing of 302021 message code. {pull}14519[14519]
- Fix filebeat azure dashboards, event category should be `Alert`. {pull}14668[14668]
- Fixed dashboard for Cisco ASA Firewall. {issue}15420[15420] {pull}15553[15553]
- Add shared_credential_file to cloudtrail config {issue}15652[15652] {pull}15656[15656]
- Fix s3 input with cloudtrail fileset reading json file. {issue}16374[16374] {pull}16441[16441]
- Fix merging of fileset inputs to replace paths and append processors. {pull}16450{16450}
- Add queue_url definition in manifest file for aws module. {pull}16640{16640}
- Fixed various Cisco FTD parsing issues. {issue}16863[16863] {pull}16889[16889]
- Fix default index pattern in IBM MQ filebeat dashboard. {pull}17146[17146]
- Fix `elasticsearch.gc` fileset to not collect _all_ logs when Elasticsearch is running in Docker. {issue}13164[13164] {issue}16583[16583] {pull}17164[17164]
- Fixed a mapping exception when ingesting CEF logs that used the spriv or dpriv extensions. {issue}17216[17216] {pull}17220[17220]
- Fixed a mapping exception when ingesting Logstash plain logs (7.4+) with pipeline ids containing non alphanumeric chars. {issue}17242[17242] {pull}17243[17243]
- Fixed MySQL slowlog module causing "regular expression has redundant nested repeat operator" warning in Elasticsearch. {issue}17086[17086] {pull}17156[17156]
- CEF: Fixed decoding errors caused by trailing spaces in messages. {pull}17253[17253]
- Fixed activemq module causing "regular expression has redundant nested repeat operator" warning in Elasticsearch. {pull}17428[17428]
- Fix issue 17734 to retry on rate-limit error in the Filebeat httpjson input. {issue}17734[17734] {pull}17735[17735]
- Remove migrationVersion map 7.7.0 reference from Kibana dashboard file to fix backward compatibility issues. {pull}17425[17425]
- Fixed `cloudfoundry.access` to have the correct `cloudfoundry.app.id` contents. {pull}17847[17847]
- Fixing `ingress_controller.` fields to be of type keyword instead of text. {issue}17834[17834]
- Fixed typo in log message. {pull}17897[17897]
- Unescape file name from SQS message. {pull}18370[18370]
- Improve cisco asa and ftd pipelines' failure handler to avoid mapping temporary fields. {issue}18391[18391] {pull}18392[18392]
- Fix source.address not being set for nginx ingress_controller {pull}18511[18511]
- Fix PANW module wrong mappings for bytes and packets counters. {issue}18522[18522] {pull}18525[18525]
- Fixed ingestion of some Cisco ASA and FTD messages when a hostname was used instead of an IP for NAT fields. {issue}14034[14034] {pull}18376[18376]
- Fix `o365.audit` failing to ingest events when ip address is surrounded by square brackets. {issue}18587[18587] {pull}18591[18591]
- Fix `o365` module ignoring `var.api` settings. {pull}18948[18948]
- Fix improper nesting of session_issuer object in aws cloudtrail fileset. {issue}18894[18894] {pull}18915[18915]
- Fix Cisco ASA ASA 3020** and 106023 messages {pull}17964[17964]
- Add missing `default_field: false` to aws filesets fields.yml. {pull}19568[19568]
- Fix bug with empty filter values in system/service {pull}19812[19812]
- Fix S3 input to trim delimiter /n from each log line. {pull}19972[19972]
- Fix auditd module syscall table for ppc64 and ppc64le. {pull}20052[20052]
- Fix Filebeat OOMs on very long lines {issue}19500[19500], {pull}19552[19552]
- Fix s3 input parsing json file without expand_event_list_from_field. {issue}19902[19902] {pull}19962[19962]
- Ignore missing in Zeek module when dropping unecessary fields. {pull}19984[19984]
- Fix s3 input parsing json file without expand_event_list_from_field. {issue}19902[19902] {pull}19962[19962] {pull}20370[20370]
- Fix millisecond timestamp normalization issues in CrowdStrike module {issue}20035[20035], {pull}20138[20138]
- Fix support for message code 106100 in Cisco ASA and FTD. {issue}19350[19350] {pull}20245[20245]
- Fix event.outcome logic for azure/siginlogs fileset {pull}20254[20254]
- Fix `fortinet` setting `event.timezone` to the system one when no `tz` field present {pull}20273[20273]
- Fix `okta` geoip lookup in pipeline for `destination.ip` {pull}20454[20454]
- Fix mapping exception in the `googlecloud/audit` dataset pipeline. {issue}18465[18465] {pull}20465[20465]
- Fix `cisco` asa and ftd parsing of messages 106102 and 106103. {pull}20469[20469]
- Improve validation checks for Azure configuration {issue}20369[20369] {pull}20389[20389]
- Fix event.kind for system/syslog pipeline {issue}20365[20365] {pull}20390[20390]
- Fix event.type for zeek/ssl and duplicate event.category for zeek/connection {pull}20696[20696]
- Fix long registry migration times. {pull}20717[20717] {issue}20705[20705]
- Fix event types and categories in auditd module to comply with ECS {pull}20652[20652]
- Update documentation in the azure module filebeat. {pull}20815[20815]
- Remove wrongly mapped `tls.client.server_name` from `fortinet/firewall` fileset. {pull}20983[20983]
- Fix an error updating file size being logged when EOF is reached. {pull}21048[21048]
- Fix error when processing AWS Cloudtrail Digest logs. {pull}21086[21086] {issue}20943[20943]
- Provide backwards compatibility for the `set` processor when Elasticsearch is less than 7.9.0. {pull}20908[20908]
- Handle multiple upstreams in ingress-controller. {pull}21215[21215]
- Provide backwards compatibility for the `append` processor when Elasticsearch is less than 7.10.0. {pull}21159[21159]
- Fix checkpoint module when logs contain time field. {pull}20567[20567]
- Add field limit check for AWS Cloudtrail flattened fields. {pull}21388[21388] {issue}21382[21382]
- Fix syslog RFC 5424 parsing in the CheckPoint module. {pull}21854[21854]
- Fix incorrect connection state mapping in zeek connection pipeline. {pull}22151[22151] {issue}22149[22149]
- Fix missing variable when loading aws pipelines. {pull}22645[22645]
- Drop aws.vpcflow.pkt_srcaddr and aws.vpcflow.pkt_dstaddr when equal to "-". {pull}22721[22721] {issue}22716[22716]

*Heartbeat*

- Fixed excessive memory usage introduced in 7.5 due to over-allocating memory for HTTP checks. {pull}15639[15639]
- Fixed scheduler shutdown issues which would in rare situations cause a panic due to semaphore misuse. {pull}16397[16397]
- Fixed TCP TLS checks to properly validate hostnames, this broke in 7.x and only worked for IP SANs. {pull}17549[17549]

*Heartbeat*

- The `service_name` monitor option is being replaced with `service.name` which is more correct. We will support the old option till 8.0. {pull}20330[20330]
- Fix exit on monitors with `enabled: false` {pull}22829[22829]

*Journalbeat*


*Metricbeat*

- Fix checking tagsFilter using length in cloudwatch metricset. {pull}14525[14525]
- Fixed bug with `elasticsearch/cluster_stats` metricset not recording license expiration date correctly. {issue}14541[14541] {pull}14591[14591]
- Log bulk failures from bulk API requests to monitoring cluster. {issue}14303[14303] {pull}14356[14356]
- Fixed bug with `elasticsearch/cluster_stats` metricset not recording license ID in the correct field. {pull}14592[14592]
- Change lookup_fields from metricset.host to service.address {pull}15883[15883]
- Fix skipping protocol scheme by light modules. {pull}16205[pull]
- Made `logstash-xpack` module once again have parity with internally-collected Logstash monitoring data. {pull}16198[16198]
- Revert changes in `docker` module: add size flag to docker.container. {pull}16600[16600]
- Fix detection and logging of some error cases with light modules. {pull}14706[14706]
- Dynamically choose a method for the system/service metricset to support older linux distros. {pull}16902[16902]
- Reduce memory usage in `elasticsearch/index` metricset. {issue}16503[16503] {pull}16538[16538]
- Fix issue in Jolokia module when mbean contains multiple quoted properties. {issue}17375[17375] {pull}17374[17374]
- Further revise check for bad data in docker/memory. {pull}17400[17400]
- Fix how we filter services by name in system/service {pull}17400[17400]
- Fix cloudwatch metricset missing tags collection. {issue}17419[17419] {pull}17424[17424]
- check if cpuOptions field is nil in DescribeInstances output in ec2 metricset. {pull}17418[17418]
- Fix aws.s3.bucket.name terms_field in s3 overview dashboard. {pull}17542[17542]
- Fix Unix socket path in memcached. {pull}17512[17512]
- Fix azure storage dashboards. {pull}17590[17590]
- Metricbeat no longer needs to be started strictly after Logstash for `logstash-xpack` module to report correct data. {issue}17261[17261] {pull}17497[17497]
- Fix pubsub metricset to collect all GA stage metrics from gcp stackdriver. {issue}17154[17154] {pull}17600[17600]
- Add privileged option so as mb to access data dir in Openshift. {pull}17606[17606]
- Fix "ID" event generator of Google Cloud module {issue}17160[17160] {pull}17608[17608]
- Add privileged option for Auditbeat in Openshift {pull}17637[17637]
- Fix storage metricset to allow config without region/zone. {issue}17623[17623] {pull}17624[17624]
- Fix overflow on Prometheus rates when new buckets are added on the go. {pull}17753[17753]
- Add a switch to the driver definition on SQL module to use pretty names {pull}17378[17378]
- Remove specific win32 api errors from events in perfmon. {issue}18292[18292] {pull}18361[18361]
- Remove required for region/zone and make stackdriver a metricset in googlecloud. {issue}16785[16785] {pull}18398[18398]
- Fix application_pool metricset after pdh changes. {pull}18477[18477]
- Fix tags_filter for cloudwatch metricset in aws. {pull}18524[18524]
- Fix panic on `metricbeat test modules` when modules are configured in `metricbeat.modules`. {issue}18789[18789] {pull}18797[18797]
- Fix getting gcp compute instance metadata with partial zone/region in config. {pull}18757[18757]
- Add missing network.sent_packets_count metric into compute metricset in googlecloud module. {pull}18802[18802]
- Fix compute and pubsub dashboard for googlecloud module. {issue}18962[18962] {pull}18980[18980]
- Fix crash on vsphere module when Host information is not available. {issue}18996[18996] {pull}19078[19078]
- Fix incorrect usage of hints builder when exposed port is a substring of the hint {pull}19052[19052]
- Stop counterCache only when already started {pull}19103[19103]
- Remove dedot for tag values in aws module. {issue}19112[19112] {pull}19221[19221]
- Fix mapping of service start type in the service metricset, windows module. {pull}19551[19551]
- Fix SQL module mapping NULL values as string {pull}18955[18955] {issue}18898[18898
- Modify doc for app_insights metricset to contain example of config. {pull}20185[20185]
- Add required option for `metrics` in app_insights. {pull}20406[20406]
- Groups same timestamp metric values to one event in the app_insights metricset. {pull}20403[20403]
- Updates vm_compute metricset with more info on guest metrics. {pull}20448[20448]
- Add fallback for PdhExpandWildCardPathW failing in perfmon metricset. {issue}20139[20139] {pull}20630[20630]
- Fix resource tags in aws cloudwatch metricset {issue}20326[20326]  {pull}20385[20385]
- Add support for azure light metricset app_stats. {pull}20639[20639]
- Fix ec2 disk and network metrics to use Sum statistic method. {pull}20680[20680]
- Fill cloud.account.name with accountID if account alias doesn't exist. {pull}20736[20736]
- Fix ec2 disk and network metrics to use Sum statistic method. {pull}20680[20680]
- The Kibana collector applies backoff when errored at getting usage stats {pull}20772[20772]
- Update fields.yml in the azure module, missing metrics field. {pull}20918[20918]
- The `elasticsearch/index` metricset only requests wildcard expansion for hidden indices if the monitored Elasticsearch cluster supports it. {pull}20938[20938]
- Disable Kafka metricsets based on Jolokia by default. They require a different configuration. {pull}20989[20989]
- Fix panic index out of range error when getting AWS account name. {pull}21101[21101] {issue}21095[21095]
- Handle missing counters in the application_pool metricset. {pull}21071[21071]
- Fix timestamp handling in remote_write. {pull}21166[21166]
- Fix remote_write flaky test. {pull}21173[21173]
- Visualization title fixes in aws, azure and googlecloud compute dashboards. {pull}21098[21098]
- Fix retrieving resources by ID for the azure module. {pull}21711[21711] {issue}21707[21707]
- Use timestamp from CloudWatch API when creating events. {pull}21498[21498]
- Report the correct windows events for system/filesystem {pull}21758[21758]
- Fix regular expression in windows/permfon. {pull}22146[22146] {issue}21125[21125]
- Fix azure storage event format. {pull}21845[21845]
- Fix panic in kubernetes autodiscover related to keystores {issue}21843[21843] {pull}21880[21880]
- [Kubernetes] Remove redundant dockersock volume mount {pull}22009[22009]
- Revert change to report `process.memory.rss` as `process.memory.wss` on Windows. {pull}22055[22055]
- Remove io.time from windows {pull}22237[22237]
- Stop generating NaN values from Cloud Foundry module to avoid errors in outputs. {pull}22634[22634]
- Fix `logstash` module when `xpack.enabled: true` is set from emitting redundant events. {pull}22808[22808]

*Packetbeat*


*Winlogbeat*


*Functionbeat*


==== Added

*Affecting all Beats*

- Decouple Debug logging from fail_on_error logic for rename, copy, truncate processors {pull}12451[12451]
- Allow a beat to ship monitoring data directly to an Elasticsearch monitoring cluster. {pull}9260[9260]
- Updated go-seccomp-bpf library to v1.1.0 which updates syscall lists for Linux v5.0. {pull}11394[11394]
- add_host_metadata is no GA. {pull}13148[13148]
- Add `providers` setting to `add_cloud_metadata` processor. {pull}13812[13812]
- Ensure that init containers are no longer tailed after they stop {pull}14394[14394]
- Fingerprint processor adds a new xxhash hashing algorithm {pull}15418[15418]
- Add configuration for APM instrumentation and expose the tracer trough the Beat object. {pull}17938[17938]
- Include network information by default on add_host_metadata and add_observer_metadata. {issue}15347[15347] {pull}16077[16077]
- Add `aws_ec2` provider for autodiscover. {issue}12518[12518] {pull}14823[14823]
- Add support for multiple password in redis output. {issue}16058[16058] {pull}16206[16206]
- Add support for Histogram type in fields.yml {pull}16570[16570]
- Remove experimental flag from `setup.template.append_fields` {pull}16576[16576]
- Add support for kubernetes provider to recognize namespace level defaults {pull}16321[16321]
- Add capability of enrich `container.id` with process id in `add_process_metadata` processor {pull}15947[15947]
- Update documentation for system.process.memory fields to include clarification on Windows os's. {pull}17268[17268]
- Add `urldecode` processor to for decoding URL-encoded fields. {pull}17505[17505]
- Add keystore support for autodiscover static configurations. {pull]16306[16306]
- When using the `decode_json_fields` processor, decoded fields are now deep-merged into existing event. {pull}17958[17958]
- Add keystore support for autodiscover static configurations. {pull]16306[16306]
- Add TLS support to Kerberos authentication in Elasticsearch. {pull}18607[18607]
- Add support for multiple sets of hints on autodiscover {pull}18883[18883]
- Add a configurable delay between retries when an app metadata cannot be retrieved by `add_cloudfoundry_metadata`. {pull}19181[19181]
- Added the `max_cached_sessions` option to the script processor. {pull}19562[19562]
- Add minimum cache TTL for successful DNS responses. {pull}18986[18986]
- Add support for DNS over TLS for the `dns` processor. {pull}19321[19321]
- Set index.max_docvalue_fields_search in index template to increase value to 200 fields. {issue}20215[20215]
- Add leader election for Kubernetes autodiscover. {pull}20281[20281]
- Add capability of enriching process metadata with contianer id also for non-privileged containers in `add_process_metadata` processor. {pull}19767[19767]
- Add replace_fields config option in add_host_metadata for replacing host fields. {pull}20490[20490] {issue}20464[20464]
- Add container ECS fields in kubernetes metadata. {pull}20984[20984]
- Add ingress controller dashboards. {pull}21052[21052]
- Added experimental `citrix` module. {pull}20820[20820]
- Added experimental `cyberark` module. {pull}20820[20820]
- Added experimental `proofpoint` module. {pull}20820[20820]
- Added experimental `snort` module. {pull}20820[20820]
- Added experimental `symantec` module. {pull}20820[20820]
- Added experimental dataset `barracuda/spamfirewall`. {pull}20820[20820]
- Added experimental dataset `cisco/meraki`. {pull}20820[20820]
- Added experimental dataset `f5/bigipafm`. {pull}20820[20820]
- Added experimental dataset `fortinet/fortimail`. {pull}20820[20820]
- Added experimental dataset `fortinet/fortimanager`. {pull}20820[20820]
- Added experimental dataset `juniper/netscreen`. {pull}20820[20820]
- Added experimental dataset `sophos/utm`. {pull}20820[20820]
- Add Cloud Foundry tags in related events. {pull}21177[21177]
- Cloud Foundry metadata is cached to disk. {pull}20775[20775]
- Add option to select the type of index template to load: legacy, component, index. {pull}21212[21212]
- Release `add_cloudfoundry_metadata` as GA. {pull}21525[21525]

*Auditbeat*

- Reference kubernetes manifests include configuration for auditd and enrichment with kubernetes metadata. {pull}17431[17431]
- Reference kubernetes manifests mount data directory from the host, so data persist between executions in the same node. {pull}17429[17429]
- Log to stderr when running using reference kubernetes manifests. {pull}17443[174443]
- Fix syscall kprobe arguments for 32-bit systems in socket module. {pull}17500[17500]
- Add ECS categorization info for auditd module {pull}18596[18596]

*Filebeat*


- `container` and `docker` inputs now support reading of labels and env vars written by docker JSON file logging driver. {issue}8358[8358]
- Add `index` option to all inputs to directly set a per-input index value. {pull}14010[14010]
- Add ECS tls fields to zeek:smtp,rdp,ssl and aws:s3access,elb {issue}15757[15757] {pull}15935[15936]
- Add ingress nginx controller fileset {pull}16197[16197]
- move create-[module,fileset,fields] to mage and enable in x-pack/filebeat {pull}15836[15836]
- Work on e2e ACK's for the azure-eventhub input {issue}15671[15671] {pull}16215[16215]
- Add a TLS test and more debug output to httpjson input {pull}16315[16315]
- Add an SSL config example in config.yml for filebeat MISP module. {pull}16320[16320]
- Improve ECS categorization, container & process field mappings in auditd module. {issue}16153[16153] {pull}16280[16280]
- Add cloudwatch fileset and ec2 fileset in aws module. {issue}13716[13716] {pull}16579[16579]
- Improve the decode_cef processor by reducing the number of memory allocations. {pull}16587[16587]
- Add custom string mapping to CEF module to support Forcepoint NGFW {issue}14663[14663] {pull}15910[15910]
- Add ECS related fields to CEF module {issue}16157[16157] {pull}16338[16338]
- Improve ECS categorization, host field mappings in elasticsearch module. {issue}16160[16160] {pull}16469[16469]
- Add pattern for Cisco ASA / FTD Message 734001 {issue}16212[16212] {pull}16612[16612]
- Added new module `o365` for ingesting Office 365 management activity API events. {issue}16196[16196] {pull}16386[16386]
- Add Filebeat Okta module. {pull}16362[16362]
- Add source field in k8s events {pull}17209[17209]
- Improve AWS cloudtrail field mappings {issue}16086[16086] {issue}16110[16110] {pull}17155[17155]
- Added new module `crowdstrike` for ingesting Crowdstrike Falcon streaming API endpoint event data. {pull}16988[16988]
- Move azure-eventhub input to GA. {issue}15671[15671] {pull}17313[17313]
- Added documentation for running Filebeat in Cloud Foundry. {pull}17275[17275]
- Added access_key_id, secret_access_key and session_token into aws module config. {pull}17456[17456]
- Release Google Cloud module as GA. {pull}17511[17511]
- Update filebeat httpjson input to support pagination via Header and Okta module. {pull}16354[16354]
- Added new Checkpoint Syslog filebeat module. {pull}17682[17682]
- Added Unix stream socket support as an input source and a syslog input source. {pull}17492[17492]
- Added new Fortigate Syslog filebeat module. {pull}17890[17890]
- Added an input option `publisher_pipeline.disable_host` to disable `host.name`
from being added to events by default. {pull}18159[18159]
- Change the `json.*` input settings implementation to merge parsed json objects with existing objects in the event instead of fully replacing them. {pull}17958[17958]
- Added http_endpoint input{pull}18298[18298]
- Add support for array parsing in azure-eventhub input. {pull}18585[18585]
- Added `observer.vendor`, `observer.product`, and `observer.type` to PANW module events. {pull}18223[18223]
- Improve ECS categorization field mappings in coredns module. {issue}16159[16159] {pull}18424[18424]
- Improve ECS categorization field mappings in envoyproxy module. {issue}16161[16161] {pull}18395[18395]
- Improve ECS categorization field mappings in cisco module. {issue}16028[16028] {pull}18537[18537]
- Add geoip AS lookup & improve ECS categorization in aws cloudtrail fileset. {issue}18644[18644] {pull}18958[18958]
- Improved performance of PANW sample dashboards. {issue}19031[19031] {pull}19032[19032]
- Add support for v1 consumer API in Cloud Foundry input, use it by default. {pull}19125[19125]
- Add new mode to multiline reader to aggregate constant number of lines {pull}18352[18352]
- Changed the panw module to pass through (rather than drop) message types other than threat and traffic. {issue}16815[16815] {pull}19375[19375]
- Improve ECS categorization field mappings in traefik module. {issue}16183[16183] {pull}19379[19379]
- Improve ECS categorization field mappings in azure module. {issue}16155[16155] {pull}19376[19376]
- Add text & flattened versions of fields with unknown subfields in aws cloudtrail fileset. {issue}18866[18866] {pull}19121[19121]
- Added Microsoft Defender ATP Module. {issue}17997[17997] {pull}19197[19197]
- Add experimental dataset tomcat/log for Apache TomCat logs {pull}19713[19713]
- Add experimental dataset netscout/sightline for Netscout Arbor Sightline logs {pull}19713[19713]
- Add experimental dataset barracuda/waf for Barracuda Web Application Firewall logs {pull}19713[19713]
- Add experimental dataset f5/bigipapm for F5 Big-IP Access Policy Manager logs {pull}19713[19713]
- Add experimental dataset bluecoat/director for Bluecoat Director logs {pull}19713[19713]
- Add experimental dataset cisco/nexus for Cisco Nexus logs {pull}19713[19713]
- Add experimental dataset citrix/virtualapps for Citrix Virtual Apps logs {pull}19713[19713]
- Add experimental dataset cylance/protect for Cylance Protect logs {pull}19713[19713]
- Add experimental dataset fortinet/clientendpoint for Fortinet FortiClient Endpoint Protection logs {pull}19713[19713]
- Add experimental dataset imperva/securesphere for Imperva Secure Sphere logs {pull}19713[19713]
- Add experimental dataset infoblox/nios for Infoblox Network Identity Operating System logs {pull}19713[19713]
- Add experimental dataset juniper/junos for Juniper Junos OS logs {pull}19713[19713]
- Add experimental dataset kaspersky/av for Kaspersky Anti-Virus logs {pull}19713[19713]
- Add experimental dataset microsoft/dhcp for Microsoft DHCP Server logs {pull}19713[19713]
- Add experimental dataset tenable/nessus_security for Tenable Nessus Security Scanner logs {pull}19713[19713]
- Add experimental dataset rapid7/nexpose for Rapid7 Nexpose logs {pull}19713[19713]
- Add experimental dataset radware/defensepro for Radware DefensePro logs {pull}19713[19713]
- Add experimental dataset sonicwall/firewall for Sonicwall Firewalls logs {pull}19713[19713]
- Add experimental dataset squid/log for Squid Proxy Server logs {pull}19713[19713]
- Add experimental dataset zscaler/zia for Zscaler Internet Access logs {pull}19713[19713]
- Added DNS response IP addresses to `related.ip` in Suricata module. {pull}22291[22291]

*Heartbeat*


*Heartbeat*

*Journalbeat*

- Added an `id` config option to inputs to allow running multiple inputs on the
same journal. {pull}18467[18467]

*Metricbeat*

- Move the windows pdh implementation from perfmon to a shared location in order for future modules/metricsets to make use of. {pull}15503[15503]
- Add DynamoDB AWS Metricbeat light module {pull}15097[15097]
- Add IBM MQ light-weight Metricbeat module {pull}15301[15301]
- Add mixer metricset for Istio Metricbeat module {pull}15696[15696]
- Add mesh metricset for Istio Metricbeat module{pull}15535[15535]
- Add pilot metricset for Istio Metricbeat module {pull}15761[15761]
- Add galley metricset for Istio Metricbeat module {pull}15857[15857]
- Add `key/value` mode for SQL module. {issue}15770[15770] {pull]15845[15845]
- Add support for Unix socket in Memcached metricbeat module. {issue}13685[13685] {pull}15822[15822]
- Add `up` metric to prometheus metrics collected from host {pull}15948[15948]
- Add citadel metricset for Istio Metricbeat module {pull}15990[15990]
- Add collecting AuroraDB metrics in rds metricset. {issue}14142[14142] {pull}16004[16004]
- Add database_account azure metricset. {issue}15758[15758]
- Add Load Balancing metricset to GCP {pull}15559[15559]
- Add OpenMetrics Metricbeat module {pull}16596[16596]
- Add system/users metricset as beta {pull}16569[16569]
- Add additional cgroup fields to docker/diskio{pull}16638[16638]
- Add Prometheus remote write endpoint {pull}16609[16609]
- Add support for CouchDB v2 {issue}16352[16352] {pull}16455[16455]
- Release Zookeeper/connection module as GA. {issue}14281[14281] {pull}17043[17043]
- Add dashboard for pubsub metricset in googlecloud module. {pull}17161[17161]
- Replace vpc metricset into vpn, transitgateway and natgateway metricsets. {pull}16892[16892]
- Use Elasticsearch histogram type to store Prometheus histograms {pull}17061[17061]
- Allow to rate Prometheus counters when scraping them {pull}17061[17061]
- Release Oracle module as GA. {issue}14279[14279] {pull}16833[16833]
- Add Storage metricsets to GCP module {pull}15598[15598]
- Release vsphere module as GA. {issue}15798[15798] {pull}17119[17119]
- Add PubSub metricset to Google Cloud Platform module {pull}15536[15536]
- Add final tests and move label to GA for the azure module in metricbeat. {pull}17319[17319]
- Added documentation for running Metricbeat in Cloud Foundry. {pull}17275[17275]
- Reference kubernetes manifests mount data directory from the host when running metricbeat as daemonset, so data persist between executions in the same node. {pull}17429[17429]
- Stack Monitoring modules now auto-configure required metricsets when `xpack.enabled: true` is set. {issue}16471[[16471] {pull}17609[17609]
- Add aggregation aligner as a config parameter for googlecloud stackdriver metricset. {issue}17141[[17141] {pull}17719[17719]
- Stack Monitoring modules now auto-configure required metricsets when `xpack.enabled: true` is set. {issue}16471[[16471] {pull}17609[17609]
- Collect new `bulk` indexing metrics from Elasticsearch when `xpack.enabled:true` is set. {issue} {pull}17992[17992]
- Remove requirement to connect as sysdba in Oracle module {issue}15846[15846] {pull}18182[18182]
- Update MSSQL module to fix some SSPI authentication and add brackets to USE statements {pull}17862[17862]]
- Add client address to events from http server module {pull}18336[18336]
- Add memory metrics into compute googlecloud. {pull}18802[18802]
- Add Tomcat overview dashboard {pull}14026[14026]
- Add support for v1 consumer API in Cloud Foundry module, use it by default. {pull}19268[19268]
- The `elasticsearch/index` metricset now collects metrics for hidden indices as well. {issue}18639[18639] {pull}18703[18703]
- Adds support for app insights metrics in the azure module. {issue}18570[18570] {pull}18940[18940]
- Added cache and connection_errors metrics to status metricset of MySQL module {issue}16955[16955] {pull}19844[19844]

*Packetbeat*

`host` metadata fields when processing network data from network tap or mirror
port. {pull}19209[19209]


*Functionbeat*

- Add monitoring info about triggered functions. {pull}14876[14876]
- Add Google Cloud Platform support. {pull}13598[13598]
- Add basic ECS categorization and `cloud` fields. {pull}19174[19174]
- Add support for parallelization factor for kinesis. {pull}20727[20727]

*Winlogbeat*

- Set process.command_line and process.parent.command_line from Sysmon Event ID 1. {pull}17327[17327]
- Add support for event IDs 4673,4674,4697,4698,4699,4700,4701,4702,4768,4769,4770,4771,4776,4778,4779,4964 to the Security module {pull}17517[17517]
- Add registry and code signature information and ECS categorization fields for sysmon module {pull}18058[18058]

*Elastic Log Driver*
- Add support for `docker logs` command {pull}19531[19531]

==== Deprecated

*Affecting all Beats*

*Filebeat*


*Heartbeat*

*Journalbeat*

*Metricbeat*


*Packetbeat*

*Winlogbeat*

*Functionbeat*

==== Known Issue

*Journalbeat*



