Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changing lodash #74539

Closed
Closed

Conversation

brittanyjoiner15
Copy link
Contributor

@brittanyjoiner15 brittanyjoiner15 commented Aug 6, 2020

Resolves #73937

Updated all instances of import { x } from 'lodash' to import x from 'lodash/x'

Used this codemod

@brittanyjoiner15 brittanyjoiner15 requested review from a team as code owners August 6, 2020 14:58
@brittanyjoiner15 brittanyjoiner15 self-assigned this Aug 6, 2020
@botelastic botelastic bot added the Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) label Aug 6, 2020
Copy link
Contributor

@watson watson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from a security perspective 👍

There seem to be a few places missing though, e.g:

import { get, mapValues, map } from 'lodash';

Have those been left out on purpose?

@brittanyjoiner15
Copy link
Contributor Author

Just realized it's not just the mapValues :) I'll go find the other lodash ones and update and then move out of draft.

@brittanyjoiner15 brittanyjoiner15 marked this pull request as ready for review August 6, 2020 16:10
@brittanyjoiner15 brittanyjoiner15 requested a review from a team as a code owner August 6, 2020 16:10
@brittanyjoiner15
Copy link
Contributor Author

@watson i realized i only did mapvalues and not all the other lodash ones. I've since updated and ran code mod to do them all!

Copy link
Contributor

@watson watson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from a security perspective 👍

I think it would be nice if we could add a linting rule in .eslintrc.js that forbids us to import directly from lodash within a public directory. I can easily see these types of imports sneaking in again, in which case all your good work is for nothing, so it would be good to guard against it.

Of the top of my head I think it would be something like this:

    {
      files: ['**/public/**/*.{js,mjs,ts,tsx}'],
      rules: {
        'no-restricted-imports': [
          2,
          {
            paths: [
              {
                name: 'lodash',
                message: 'Please import specific method as a file instead, e.g. "lodash/get"',
              },
            ],
          },
        ],
      },
    },

@botelastic botelastic bot added the Team:APM All issues that need APM UI Team support label Aug 10, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/apm-ui (Team:apm)

@brittanyjoiner15
Copy link
Contributor Author

Per this discussion on Slack, we're going to do this for the whole repo. I'll also work on adding the eslint rule above. Will be working on this this week!

@brittanyjoiner15 brittanyjoiner15 requested review from a team as code owners August 10, 2020 20:38
@brittanyjoiner15 brittanyjoiner15 requested a review from a team August 10, 2020 20:38
@brittanyjoiner15 brittanyjoiner15 requested review from a team as code owners August 10, 2020 20:38
@brittanyjoiner15 brittanyjoiner15 requested a review from a team August 10, 2020 20:38
@brittanyjoiner15 brittanyjoiner15 requested review from a team as code owners August 10, 2020 20:38
@tylersmalley
Copy link
Contributor

@pgayvallet @watson @dgieselaar

For SIEM, that was done here.

I personally prefer we write the code as we expect it to behave instead of relying on transformations. It really makes debugging a nightmare. Do you all disagree?

This change might need to be broken up and a few plugins at a time. We have done this in the past and added exceptions to the ESLint rule and pointed to issues for the plugins to remove the exception and become compliant.

@mistic
Copy link
Member

mistic commented Aug 18, 2020

I agree with @tylersmalley on this one. Although I'm not sure what will be the penalty impact of transform this at compile time, our lodash usage is really wide and I would prefer to have the code explicitly behave as expected. Another thing to consider is that in a future lodash major upgrade it would be easier to find the places with lodash usages we need to convert because we can just search for lodash/{function_name}. Finally, I'm not sure if the plugin is working as expected right now: lodash/babel-plugin-lodash#236. If we endup deciding to go through the transformation route, babel-plugin-transform-imports probably would be a better alternative as while it does not remap default imports it works as expected over named imports.

@brittanyjoiner15
Copy link
Contributor Author

brittanyjoiner15 commented Aug 24, 2020

Nathan and I are doing this...

In summary, what we need to watch out for:

  • Make sure no extra new lines are added
  • Only edit in /public,/common, and /packages

to do:

  • Revert to master
  • write and run ESlint rule to get list of files
  • do find import _, (\{ [a-zA-Z\s,]+ \}) from 'lodash' and replace with import $1 from 'lodash'
  • run codemod on tht list of files
  • fix spaces
  • fix aliases
  • turn back on prettier rules

@dgieselaar
Copy link
Member

dgieselaar commented Aug 24, 2020 via email

@dgieselaar
Copy link
Member

dgieselaar commented Aug 24, 2020 via email

@brittanyjoiner15
Copy link
Contributor Author

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

merge conflict between base and head

@brittanyjoiner15
Copy link
Contributor Author

brittanyjoiner15 commented Sep 4, 2020

We're close! Just need to fetch the most recent and do this again to avoid conflicts.

  • turn off prettier
  • get list of files ( $yarn lint:es --format unix > output and then cat Output | cut -d ":" -f 1 > listOfFiles )
  • run the codemod (jscodeshift --stdin --extensions=js,jsx,ts,tsx -t modular-lodash.js --parser=tsx < listOfFiles)
  • run the regex to fix new line spaces (find: import(.*)lodash(.*)\n\nimport(.*) replace import$1lodash$2\nimport$3
  • run the regex to fix aliases (git diff upstream/master | code - and search for import.*\sas\s.*lodash )
  • turn prettier back on

@kibanamachine
Copy link
Contributor

kibanamachine commented Sep 4, 2020

💔 Build Failed

Failed CI Steps


Test Failures

X-Pack Jest Tests.x-pack/plugins/rollup/public/test/client_integration.Create Rollup Job, step 1: Logistics form validations index pattern should not allow spaces

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches


Stack Trace

Error: expect(received).toContain(expected) // indexOf

Expected value: "Remove the spaces from your index pattern."
Received array: []
    at Object.it (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/rollup/public/test/client_integration/job_create_logistics.test.js:98:42)

X-Pack Jest Tests.x-pack/plugins/rollup/public/test/client_integration.Create Rollup Job, step 1: Logistics form validations index pattern should not allow an unknown index pattern

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches


Stack Trace

Error: expect(received).toContain(expected) // indexOf

Expected value: "Index pattern doesn't match any indices."
Received array: []
    at Object.it (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/rollup/public/test/client_integration/job_create_logistics.test.js:105:42)

X-Pack Jest Tests.x-pack/plugins/rollup/public/test/client_integration.Create Rollup Job, step 1: Logistics form validations index pattern should not allow an index pattern without time fields

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches


Stack Trace

Error: expect(received).toContain(expected) // indexOf

Expected value: "Index pattern must match indices that contain time fields."
Received array: []
    at Object.it (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/rollup/public/test/client_integration/job_create_logistics.test.js:112:42)

and 83 more failures, only showing the first 3.

Build metrics

@kbn/optimizer bundle module count

id value diff baseline
advancedSettings 97 +44 53
apm 1173 +231 942
beatsManagement 470 +163 307
canvas 1388 +185 1203
charts 456 +188 268
console 292 +129 163
core 717 +156 561
crossClusterReplication 217 +54 163
dashboard 303 +127 176
dashboardEnhanced 149 +120 29
dashboardMode 29 +22 7
data 778 +236 542
dataEnhanced 175 +157 18
devTools 155 +135 20
discover 438 +180 258
embeddable 212 +120 92
esUiShared 299 +72 227
expressions 302 +193 109
graph 284 +125 159
grokdebugger 133 +94 39
indexLifecycleManagement 313 +147 166
indexManagement 649 +149 500
indexPatternManagement 377 +161 216
infra 1232 +144 1088
inspector 65 -2 67
kibanaLegacy 209 +75 134
kibanaReact 435 +141 294
kibanaUtils 364 +175 189
lens 436 +181 255
logstash 380 +185 195
management 180 +135 45
ml 1407 +34 1373
monitoring 797 +180 617
navigation 46 +20 26
observability 234 +139 95
painlessLab 54 +10 44
regionMap 439 +188 251
remoteClusters 240 +95 145
reporting 96 +52 44
rollup 361 +178 183
savedObjects 105 +57 48
savedObjectsManagement 251 +132 119
searchprofiler 196 +120 76
security 575 +120 455
securitySolution 1958 +40 1918
snapshotRestore 304 +136 168
spaces 312 +120 192
tileMap 448 +191 257
transform 455 +135 320
triggers_actions_ui 395 +171 224
uiActions 158 +124 34
upgradeAssistant 217 +98 119
uptime 543 +36 507
visTypeMarkdown 446 +188 258
visTypeMetric 450 +190 260
visTypeTable 469 +192 277
visTypeTagcloud 449 +188 261
visTypeTimelion 491 +211 280
visTypeTimeseries 522 +54 468
visTypeVega 499 +188 311
visTypeVislib 717 +210 507
visualizations 260 +125 135
visualize 514 +199 315
watcher 340 +113 227
total +8456

async chunks size

id value diff baseline
advancedSettings 76.8KB +11.2KB 65.5KB
apm 4.7MB +1.6KB 4.7MB
beatsManagement 570.7KB +161.4KB 409.3KB
canvas 1.4MB +15.6KB 1.4MB
charts 44.8KB +2.0B 44.8KB
console 1.2MB +48.4KB 1.1MB
crossClusterReplication 466.1KB +3.6KB 462.5KB
devTools 6.5KB +7.0B 6.5KB
discover 523.4KB +92.8KB 430.6KB
graph 1.4MB +43.3KB 1.4MB
grokdebugger 586.4KB -111.5KB 697.9KB
indexLifecycleManagement 268.0KB +9.7KB 258.3KB
indexManagement 1.7MB +147.5KB 1.6MB
indexPatternManagement 723.4KB -83.2KB 806.6KB
infra 3.6MB +58.1KB 3.5MB
kibanaLegacy 147.1KB +49.0B 147.1KB
logstash 217.3KB +1.2KB 216.1KB
management 83.6KB -93.3KB 176.8KB
ml 8.2MB +28.4KB 8.2MB
monitoring 1.2MB +78.6KB 1.1MB
observability 220.0KB -90.3KB 310.3KB
painlessLab 46.6KB -137.6KB 184.2KB
regionMap 44.8KB +1.0B 44.8KB
remoteClusters 264.6KB -110.8KB 375.4KB
rollup 350.8KB +20.3KB 330.5KB
savedObjectsManagement 157.2KB -11.7KB 168.9KB
searchprofiler 756.5KB +41.6KB 714.8KB
security 1.2MB +147.4KB 1.0MB
securitySolution 7.2MB +10.9KB 7.2MB
snapshotRestore 665.3KB -92.8KB 758.2KB
spaces 45.9KB +122.0B 45.8KB
tileMap 44.8KB +1.0B 44.8KB
transform 615.4KB -89.1KB 704.5KB
triggers_actions_ui 834.1KB +97.6KB 736.6KB
upgradeAssistant 219.9KB -102.5KB 322.3KB
uptime 1.4MB -129.5KB 1.6MB
visTypeMarkdown 40.7KB +2.0B 40.7KB
visTypeMetric 38.8KB +1.0B 38.8KB
visTypeTable 45.4KB +33.0B 45.4KB
visTypeTagcloud 38.8KB +2.0B 38.8KB
visTypeTimelion 38.8KB +1.0B 38.8KB
visTypeTimeseries 1.9MB +145.7KB 1.8MB
visTypeVega 1.4MB +183.0B 1.4MB
visTypeVislib 38.8KB +2.0B 38.8KB
visualize 768.6KB +74.8KB 693.8KB
watcher 644.4KB +42.5KB 602.0KB
total +230.4KB

page load bundle size

id value diff baseline
advancedSettings 844.2KB -136.5KB 980.7KB
apm 40.7KB +76.0B 40.6KB
beatsManagement 488.5KB -93.1KB 581.6KB
canvas 1.4MB +63.9KB 1.3MB
charts 944.6KB +71.0KB 873.6KB
console 30.4KB +1.0B 30.4KB
core 1.2MB +66.8KB 1.2MB
crossClusterReplication 64.5KB -127.6KB 192.1KB
dashboard 747.1KB +44.9KB 702.2KB
dashboardEnhanced 88.4KB -97.3KB 185.6KB
dashboardMode 14.9KB -135.3KB 150.2KB
data 1.5MB +107.5KB 1.4MB
dataEnhanced 91.0KB -87.1KB 178.1KB
devTools 72.8KB -93.4KB 166.2KB
discover 232.9KB +3.6KB 229.3KB
embeddable 333.9KB -96.6KB 430.5KB
esUiShared 1018.7KB +29.6KB 989.2KB
expressions 276.5KB -70.1KB 346.6KB
grokdebugger 11.4KB +1.0B 11.4KB
indexLifecycleManagement 139.7KB -98.2KB 237.9KB
indexManagement 154.7KB -93.3KB 248.1KB
indexPatternManagement 135.4KB +1.0B 135.4KB
infra 136.0KB -139.7KB 275.6KB
inspector 135.7KB -142.8KB 278.4KB
kibanaLegacy 275.8KB +28.2KB 247.7KB
kibanaReact 551.3KB -90.8KB 642.1KB
kibanaUtils 392.6KB -77.8KB 470.4KB
lens 935.1KB +68.8KB 866.3KB
logstash 107.7KB -72.3KB 180.1KB
management 29.5KB +1.0B 29.5KB
ml 433.8KB -142.3KB 576.1KB
navigation 28.9KB -135.8KB 164.7KB
observability 48.9KB +1.0B 48.9KB
painlessLab 161.0KB -40.0B 161.0KB
regionMap 903.4KB +70.9KB 832.4KB
remoteClusters 35.4KB +1.0B 35.4KB
reporting 306.0KB +15.2KB 290.8KB
rollup 129.7KB -93.3KB 223.1KB
savedObjects 254.5KB +17.1KB 237.4KB
savedObjectsManagement 144.3KB -72.1KB 216.4KB
searchprofiler 50.9KB +1.0B 50.9KB
security 200.2KB -105.0KB 305.2KB
securitySolution 674.5KB -132.2KB 806.7KB
snapshotRestore 61.8KB +1.0B 61.8KB
spaces 455.2KB +42.4KB 412.8KB
tileMap 916.0KB +71.9KB 844.1KB
triggers_actions_ui 121.1KB -142.3KB 263.5KB
uiActions 262.3KB +43.7KB 218.6KB
upgradeAssistant 64.7KB +1.0B 64.7KB
visTypeMarkdown 627.0KB +70.9KB 556.1KB
visTypeMetric 652.5KB +71.6KB 580.9KB
visTypeTable 677.9KB +73.0KB 604.9KB
visTypeTagcloud 908.2KB +70.9KB 837.3KB
visTypeTimelion 794.9KB +82.8KB 712.1KB
visTypeTimeseries 157.1KB -123.5KB 280.7KB
visTypeVega 732.6KB +70.9KB 661.7KB
visTypeVislib 1.4MB +89.5KB 1.3MB
visualizations 461.2KB +53.9KB 407.4KB
watcher 28.9KB +1.0B 28.9KB
total -1.2MB

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@stacey-gammon
Copy link
Contributor

is this PR old and should be closed? 7.9.2 FF is today... also this doesn't seem like it should go into a patch release. Going to remove the label!

@smith
Copy link
Contributor

smith commented Sep 18, 2020

is this PR old and should be closed? 7.9.2 FF is today... also this doesn't seem like it should go into a patch release. Going to remove the label!

Thanks for removing the label. It's still in progress but it's a big hairy change.

@smith
Copy link
Contributor

smith commented Sep 24, 2020

Closing in favor of #78156.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Embedding Embedding content via iFrame Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) release_note:enhancement Team:APM All issues that need APM UI Team support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[APM] Use default imports from lodash