Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM] Adds Signals Histogram #53742

Merged
merged 13 commits into from
Jan 10, 2020
Merged

[SIEM] Adds Signals Histogram #53742

merged 13 commits into from
Jan 10, 2020

Conversation

spong
Copy link
Member

@spong spong commented Dec 21, 2019

Summary

Detection Engine Meta Issue: #50405

This PR adds the Signals Histogram component for use on the main Detection Engine page, Rule Details page, and the newly designed Overview page.

Out of the box configuration includes an EuiSelect for stacking by the following:

  • Risk Scores
  • Severities
  • Event Actions
  • Event Categories
  • Host Names
  • Rule Types
  • Rules
  • Users
  • Destination IPs
  • Source IPs

Additional configuration properties are available to configure the component as needed depending on where it will be displayed (e.g. no Stack By option on Overview, filter to specific rule_id on Rule Details, etc):

interface SignalsHistogramPanelProps {
  defaultStackByOption?: SignalsHistogramOption;
  filters?: esFilters.Filter[];
  from: number;
  query?: Query;
  legendPosition?: 'left' | 'right' | 'bottom' | 'top';
  loadingInitial?: boolean;
  showLinkToSignals?: boolean;
  showTotalSignalsCount?: boolean;
  stackByOptions?: SignalsHistogramOption[];
  title?: string;
  to: number;
  updateDateRange: (min: number, max: number) => void;
}
Light Theme:

de_hist_light

Dark Theme:

de_histogram_dark

Overview:

Example props for overview impl:

<SignalsHistogramPanel
  filters={filters}
  from={from}
  loadingInitial={loading}
  query={query}
  showTotalSignalsCount={true}
  showLinkToSignals={true}
  defaultStackByOption={{
    text: 'Signals count by MITRE ATT&CK category',
    value: 'signal.rule.threats',
  }}
  legendPosition={'right'}
  to={to}
  title="Signals count by MITRE ATT&CK category"
  updateDateRange={updateDateRangeCallback}
/>

image

Note @andrew-goldstein @angorayc @MichaelMarcialis -- looks like the MITRE ATT&CK Tactics are stored as a nested object in signal.rule.threat, so we may have to do some finangling to get it to show on the histogram.

e.g. format:

{
  "framework": "MITRE ATT&CK",
  "tactic": {
    "id": "TA0010",
    "reference": "https://attack.mitre.org/tactics/TA0010",
    "name": "Exfiltration"
  },
  "techniques": [
    {
      "id": "T1002",
      "name": "Data Compressed",
      "reference": "https://attack.mitre.org/techniques/T1002"
    }
  ]
}

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

For maintainers

@spong spong added Team:SIEM release_note:skip Skip the PR/issue when compiling release notes labels Dec 21, 2019
@spong spong self-assigned this Dec 21, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

signalsByGrouping: {
terms: {
field: stackByField,
missing: stackByField.endsWith('.ip') ? '0.0.0.0' : i18n.ALL_OTHERS,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that we still have i18n in histograms' label!

Copy link
Contributor

@angorayc angorayc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the branch and all worked well. Thanks a lot for adding the dropdown list and histogram!

/>
</EuiFlexItem>
/>,
rule?.updated_by != null ? (
Copy link
Contributor

@FrankHassanabad FrankHassanabad Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing for a few reasons and the intention is not really there.

The io-ts TypeScript for updated_by and created_by both say they cannot be null but looking at the operators it does look like they can be which I think is the actual use case. So I would change the types.

The second part is that when updated_by is null it will never hit these lines below and show an updated_by by UNKNOWN but rather just show an empty string.

So which is it we want? To show nothing for updated_by when it is null or to show it but mark it as UNKNOWN?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oooh good catch. This is existing code that got reformatted from my changes, so will have to hear from @XavierM on this, but my best guess is that we'd want the latter and show UNKNOWN. Although there may be some special logic I'm missing out on for how updated_by is used?

by: rule?.updated_by ?? i18n.UNKNOWN,
date: (
<FormattedDate
value={rule?.updated_at ?? new Date().toISOString()}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the backend we always provide a updated_at and a created_at without mistake as far as I know now. So the checks for these should never happen.

However, same issues with TypeScript saying that these two can never be null/undefined. If you want to be safe and just guard against a regression bug on our side or something else, I would change the type system to reflect that and also I would not put "fake dates" such as new Date().toISOString() but rather defer to using a -- nonexistent placeholder to show that they just aren't present.

Up to you.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @XavierM -- this may be an artifact from initial development when they weren't always present?

Copy link
Contributor

@FrankHassanabad FrankHassanabad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few conversations but feel free to merge anytime.

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@spong spong merged commit 482faae into elastic:master Jan 10, 2020
@spong spong deleted the de-signals-histogram branch January 10, 2020 00:53
spong added a commit to spong/kibana that referenced this pull request Jan 10, 2020
## Summary

Detection Engine Meta Issue: elastic#50405

This PR adds the `Signals Histogram` component for use on the main `Detection Engine` page, `Rule Details` page, and the newly designed `Overview` page.

Out of the box configuration includes an `EuiSelect` for stacking by the following:
* Risk Scores
* Severities
* Event Actions
* Event Categories
* Host Names
* Rule Types
* Rules
* Users
* Destination IPs
* Source IPs

Additional configuration properties are available to configure the component as needed depending on where it will be displayed (e.g. no `Stack By` option on `Overview`, filter to specific `rule_id` on `Rule Details`, etc):

``` ts
interface SignalsHistogramPanelProps {
  defaultStackByOption?: SignalsHistogramOption;
  filters?: esFilters.Filter[];
  from: number;
  query?: Query;
  legendPosition?: 'left' | 'right' | 'bottom' | 'top';
  loadingInitial?: boolean;
  showLinkToSignals?: boolean;
  showTotalSignalsCount?: boolean;
  stackByOptions?: SignalsHistogramOption[];
  title?: string;
  to: number;
  updateDateRange: (min: number, max: number) => void;
}
```
##### Light Theme:
![de_hist_light](https://user-images.githubusercontent.com/2946766/71299977-41685800-234e-11ea-93bd-05a0c4cb6ee1.gif)

##### Dark Theme:
![de_histogram_dark](https://user-images.githubusercontent.com/2946766/71299980-45947580-234e-11ea-9d26-380bae5c4aa6.gif)


##### Overview:

Example props for overview impl:

``` jsx
<SignalsHistogramPanel
  filters={filters}
  from={from}
  loadingInitial={loading}
  query={query}
  showTotalSignalsCount={true}
  showLinkToSignals={true}
  defaultStackByOption={{
    text: 'Signals count by MITRE ATT&CK category',
    value: 'signal.rule.threats',
  }}
  legendPosition={'right'}
  to={to}
  title="Signals count by MITRE ATT&CK category"
  updateDateRange={updateDateRangeCallback}
/>
```
![image](https://user-images.githubusercontent.com/2946766/72030438-2fd7e900-3246-11ea-8404-40905ca5f85c.png)


Note @andrew-goldstein @angorayc @MichaelMarcialis -- looks like the MITRE ATT&CK Tactics are stored as a nested object in `signal.rule.threat`, so we may have to do some finangling to get it to show on the histogram. 

e.g. format:

``` json
{
  "framework": "MITRE ATT&CK",
  "tactic": {
    "id": "TA0010",
    "reference": "https://attack.mitre.org/tactics/TA0010",
    "name": "Exfiltration"
  },
  "techniques": [
    {
      "id": "T1002",
      "name": "Data Compressed",
      "reference": "https://attack.mitre.org/techniques/T1002"
    }
  ]
}
```




### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)
- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials
  * Will work with @benskelker on any specific documentation
- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~

### For maintainers

- [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
- [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
spong added a commit that referenced this pull request Jan 10, 2020
## Summary

Detection Engine Meta Issue: #50405

This PR adds the `Signals Histogram` component for use on the main `Detection Engine` page, `Rule Details` page, and the newly designed `Overview` page.

Out of the box configuration includes an `EuiSelect` for stacking by the following:
* Risk Scores
* Severities
* Event Actions
* Event Categories
* Host Names
* Rule Types
* Rules
* Users
* Destination IPs
* Source IPs

Additional configuration properties are available to configure the component as needed depending on where it will be displayed (e.g. no `Stack By` option on `Overview`, filter to specific `rule_id` on `Rule Details`, etc):

``` ts
interface SignalsHistogramPanelProps {
  defaultStackByOption?: SignalsHistogramOption;
  filters?: esFilters.Filter[];
  from: number;
  query?: Query;
  legendPosition?: 'left' | 'right' | 'bottom' | 'top';
  loadingInitial?: boolean;
  showLinkToSignals?: boolean;
  showTotalSignalsCount?: boolean;
  stackByOptions?: SignalsHistogramOption[];
  title?: string;
  to: number;
  updateDateRange: (min: number, max: number) => void;
}
```
##### Light Theme:
![de_hist_light](https://user-images.githubusercontent.com/2946766/71299977-41685800-234e-11ea-93bd-05a0c4cb6ee1.gif)

##### Dark Theme:
![de_histogram_dark](https://user-images.githubusercontent.com/2946766/71299980-45947580-234e-11ea-9d26-380bae5c4aa6.gif)


##### Overview:

Example props for overview impl:

``` jsx
<SignalsHistogramPanel
  filters={filters}
  from={from}
  loadingInitial={loading}
  query={query}
  showTotalSignalsCount={true}
  showLinkToSignals={true}
  defaultStackByOption={{
    text: 'Signals count by MITRE ATT&CK category',
    value: 'signal.rule.threats',
  }}
  legendPosition={'right'}
  to={to}
  title="Signals count by MITRE ATT&CK category"
  updateDateRange={updateDateRangeCallback}
/>
```
![image](https://user-images.githubusercontent.com/2946766/72030438-2fd7e900-3246-11ea-8404-40905ca5f85c.png)


Note @andrew-goldstein @angorayc @MichaelMarcialis -- looks like the MITRE ATT&CK Tactics are stored as a nested object in `signal.rule.threat`, so we may have to do some finangling to get it to show on the histogram. 

e.g. format:

``` json
{
  "framework": "MITRE ATT&CK",
  "tactic": {
    "id": "TA0010",
    "reference": "https://attack.mitre.org/tactics/TA0010",
    "name": "Exfiltration"
  },
  "techniques": [
    {
      "id": "T1002",
      "name": "Data Compressed",
      "reference": "https://attack.mitre.org/techniques/T1002"
    }
  ]
}
```




### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)
- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials
  * Will work with @benskelker on any specific documentation
- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~

### For maintainers

- [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
- [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
thomasneirynck pushed a commit to thomasneirynck/kibana that referenced this pull request Jan 12, 2020
## Summary

Detection Engine Meta Issue: elastic#50405

This PR adds the `Signals Histogram` component for use on the main `Detection Engine` page, `Rule Details` page, and the newly designed `Overview` page.

Out of the box configuration includes an `EuiSelect` for stacking by the following:
* Risk Scores
* Severities
* Event Actions
* Event Categories
* Host Names
* Rule Types
* Rules
* Users
* Destination IPs
* Source IPs

Additional configuration properties are available to configure the component as needed depending on where it will be displayed (e.g. no `Stack By` option on `Overview`, filter to specific `rule_id` on `Rule Details`, etc):

``` ts
interface SignalsHistogramPanelProps {
  defaultStackByOption?: SignalsHistogramOption;
  filters?: esFilters.Filter[];
  from: number;
  query?: Query;
  legendPosition?: 'left' | 'right' | 'bottom' | 'top';
  loadingInitial?: boolean;
  showLinkToSignals?: boolean;
  showTotalSignalsCount?: boolean;
  stackByOptions?: SignalsHistogramOption[];
  title?: string;
  to: number;
  updateDateRange: (min: number, max: number) => void;
}
```
##### Light Theme:
![de_hist_light](https://user-images.githubusercontent.com/2946766/71299977-41685800-234e-11ea-93bd-05a0c4cb6ee1.gif)

##### Dark Theme:
![de_histogram_dark](https://user-images.githubusercontent.com/2946766/71299980-45947580-234e-11ea-9d26-380bae5c4aa6.gif)


##### Overview:

Example props for overview impl:

``` jsx
<SignalsHistogramPanel
  filters={filters}
  from={from}
  loadingInitial={loading}
  query={query}
  showTotalSignalsCount={true}
  showLinkToSignals={true}
  defaultStackByOption={{
    text: 'Signals count by MITRE ATT&CK category',
    value: 'signal.rule.threats',
  }}
  legendPosition={'right'}
  to={to}
  title="Signals count by MITRE ATT&CK category"
  updateDateRange={updateDateRangeCallback}
/>
```
![image](https://user-images.githubusercontent.com/2946766/72030438-2fd7e900-3246-11ea-8404-40905ca5f85c.png)


Note @andrew-goldstein @angorayc @MichaelMarcialis -- looks like the MITRE ATT&CK Tactics are stored as a nested object in `signal.rule.threat`, so we may have to do some finangling to get it to show on the histogram. 

e.g. format:

``` json
{
  "framework": "MITRE ATT&CK",
  "tactic": {
    "id": "TA0010",
    "reference": "https://attack.mitre.org/tactics/TA0010",
    "name": "Exfiltration"
  },
  "techniques": [
    {
      "id": "T1002",
      "name": "Data Compressed",
      "reference": "https://attack.mitre.org/techniques/T1002"
    }
  ]
}
```




### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)
- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials
  * Will work with @benskelker on any specific documentation
- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~

### For maintainers

- [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
- [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:SIEM v7.6.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants