BREAKING(react-conformance): update test type and add new option `disableTypeTests` to disables tests that requires type information #28988

YuanboXue-Amber · 2023-08-25T10:44:31Z

isConformant build TS program from tsconfig root. It can cause out of memory issue in a large repo. But most of the tests do not require typescript info. When building fluent-like components in a large repo, there is no way to do conformance test and bypass building typescript program.

This PR adds new option disableTypeTests that disables tests requiring TS info. When disableTypeTests=true, there's no need to build TS program. Fluent-like components in a large repo can use this option to get coverage of most conformance tests.

To determine which test needs TS info, this PR updates the type of the test:

before, all tests have type:

export type ConformanceTest<TProps = {}> = (componentInfo: ComponentDoc, testInfo: IsConformantOptions<TProps>, tsProgram: ts.Program) => void

after - componentInfo and tsProgram require TS info. So testInfo is swapped to first:

export type ConformanceTest<TProps = {}> = (testInfo: IsConformantOptions<TProps>, componentInfo: ComponentDoc, tsProgram: ts.Program) => void

And tests that do not need TS info have type:

export type BaseConformanceTest<TProps = {}> = (testInfo: IsConformantOptions<TProps>) => void;

We check argument length to determine if a test requires TS info.

fabricteam · 2023-08-25T10:51:38Z

📊 Bundle size report

Unchanged fixtures

Package & Exports	Size (minified/GZIP)
`react-accordion` Accordion (including children components)	`91.384 kB` `27.926 kB`
`react-alert` Alert	`84.549 kB` `23.128 kB`
`react-avatar` Avatar	`49.738 kB` `15.631 kB`
`react-avatar` AvatarGroup	`18.584 kB` `7.448 kB`
`react-avatar` AvatarGroupItem	`64.39 kB` `19.977 kB`
`react-badge` Badge	`25.793 kB` `8.348 kB`
`react-badge` CounterBadge	`26.694 kB` `8.658 kB`
`react-badge` PresenceBadge	`24.75 kB` `8.96 kB`
`react-button` Button	`39.658 kB` `10.79 kB`
`react-button` CompoundButton	`47.013 kB` `12.284 kB`
`react-button` MenuButton	`44.085 kB` `12.041 kB`
`react-button` SplitButton	`52.111 kB` `13.612 kB`
`react-button` ToggleButton	`56.746 kB` `12.698 kB`
`react-card` Card - All	`91.249 kB` `26.409 kB`
`react-card` Card	`86.038 kB` `24.868 kB`
`react-card` CardFooter	`11.951 kB` `5.031 kB`
`react-card` CardHeader	`14.237 kB` `5.798 kB`
`react-card` CardPreview	`12.903 kB` `5.408 kB`
`react-checkbox` Checkbox	`35.614 kB` `11.771 kB`
`react-combobox` Combobox (including child components)	`90.383 kB` `29.52 kB`
`react-combobox` Dropdown (including child components)	`88.738 kB` `29.167 kB`
`react-components` react-components: Button, FluentProvider & webLightTheme	`69.517 kB` `19.64 kB`
`react-components` react-components: Accordion, Button, FluentProvider, Image, Menu, Popover	`208.327 kB` `59.386 kB`
`react-components` react-components: FluentProvider & webLightTheme	`40.713 kB` `13.509 kB`
`react-datepicker-compat` DatePicker Compat	`211.433 kB` `58.785 kB`
`react-dialog` Dialog (including children components)	`89.917 kB` `27.447 kB`
`react-divider` Divider	`19.704 kB` `7.38 kB`
`react-field` Field	`21.036 kB` `8.091 kB`
`react-image` Image	`14.62 kB` `5.869 kB`
`react-infobutton` InfoButton	`129.444 kB` `40.699 kB`
`react-infobutton` InfoLabel	`133.169 kB` `41.891 kB`
`react-input` Input	`25.955 kB` `8.81 kB`
`react-jsx-runtime` Classic Pragma	`1.072 kB` `544 B`
`react-jsx-runtime` JSX Dev Runtime	`2.775 kB` `1.29 kB`
`react-jsx-runtime` JSX Runtime	`3.293 kB` `1.529 kB`
`react-label` Label	`13.036 kB` `5.405 kB`
`react-link` Link	`15.902 kB` `6.506 kB`
`react-menu` Menu (including children components)	`139.629 kB` `43.01 kB`
`react-menu` Menu (including selectable components)	`142.365 kB` `43.541 kB`
`react-persona` Persona	`56.629 kB` `17.505 kB`
`react-popover` Popover	`118.698 kB` `37.359 kB`
`react-portal` Portal	`12.362 kB` `4.543 kB`
`react-portal-compat` PortalCompatProvider	`6.541 kB` `2.227 kB`
`react-progress` ProgressBar	`16.409 kB` `6.58 kB`
`react-provider` FluentProvider	`21.258 kB` `7.937 kB`
`react-radio` Radio	`29.318 kB` `9.699 kB`
`react-radio` RadioGroup	`14.344 kB` `5.942 kB`
`react-select` Select	`27.324 kB` `9.773 kB`
`react-slider` Slider	`36.849 kB` `12.171 kB`
`react-spinbutton` SpinButton	`35.53 kB` `11.367 kB`
`react-spinner` Spinner	`22.292 kB` `8.113 kB`
`react-switch` Switch	`31.885 kB` `10.356 kB`
`react-table` DataGrid	`156.473 kB` `43.597 kB`
`react-table` Table (Primitives only)	`42.52 kB` `13.286 kB`
`react-table` Table as DataGrid	`129.333 kB` `34.77 kB`
`react-table` Table (Selection only)	`74.617 kB` `20.089 kB`
`react-table` Table (Sort only)	`73.248 kB` `19.685 kB`
`react-tags-preview` InteractionTag	`13.85 kB` `5.626 kB`
`react-tags-preview` Tag	`29.604 kB` `9.567 kB`
`react-tags-preview` TagGroup	`72.493 kB` `21.628 kB`
`react-text` Text - Default	`15.644 kB` `6.223 kB`
`react-text` Text - Wrappers	`18.817 kB` `6.546 kB`
`react-textarea` Textarea	`30.005 kB` `10.156 kB`
`react-toast` Toast (including Toaster)	`90.782 kB` `27.171 kB`
`react-tooltip` Tooltip	`51.154 kB` `18.06 kB`

_{🤖 This report was generated against 42ed56cff2ef6c21fb2d5805d77c370dc3036ebe}

fabricteam · 2023-08-25T10:58:10Z

Perf Analysis (`@fluentui/react-components`)

No significant results to display.

All results

Scenario	Render type	Master Ticks	PR Ticks	Iterations
Avatar	mount	606	604	5000
Button	mount	304	309	5000
Field	mount	1095	1100	5000
FluentProvider	mount	699	686	5000
FluentProviderWithTheme	mount	76	79	10
FluentProviderWithTheme	virtual-rerender	69	69	10
FluentProviderWithTheme	virtual-rerender-with-unmount	77	71	10
InfoButton	mount	8	11	5000
MakeStyles	mount	843	864	50000
Persona	mount	1699	1688	5000
SpinButton	mount	1366	1340	5000

fabricteam · 2023-08-25T10:58:41Z

Perf Analysis (`@fluentui/react-northstar`)

⚠️ 1 potential perf regressions detected

Potential regressions comparing to master

Scenario	Current PR Ticks	Baseline Ticks	Ratio	Regression Analysis
ButtonMinimalPerf.default	83	88	0.94:1	analysis

Perf tests with no regressions

Scenario	Current PR Ticks	Baseline Ticks	Ratio
RefMinimalPerf.default	114	107	1.07:1
PortalMinimalPerf.default	87	82	1.06:1
ButtonSlotsPerf.default	313	298	1.05:1
CarouselMinimalPerf.default	264	251	1.05:1
LabelMinimalPerf.default	225	215	1.05:1
IconMinimalPerf.default	391	372	1.05:1
AvatarMinimalPerf.default	104	100	1.04:1
ChatDuplicateMessagesPerf.default	146	140	1.04:1
HeaderSlotsPerf.default	468	451	1.04:1
TableMinimalPerf.default	238	229	1.04:1
CheckboxMinimalPerf.default	1123	1089	1.03:1
FormMinimalPerf.default	224	217	1.03:1
GridMinimalPerf.default	191	186	1.03:1
LoaderMinimalPerf.default	197	191	1.03:1
SkeletonMinimalPerf.default	201	195	1.03:1
TextMinimalPerf.default	192	187	1.03:1
HeaderMinimalPerf.default	211	207	1.02:1
RosterPerf.default	1565	1530	1.02:1
ProviderMergeThemesPerf.default	658	645	1.02:1
StatusMinimalPerf.default	388	382	1.02:1
AttachmentSlotsPerf.default	617	611	1.01:1
CardMinimalPerf.default	306	304	1.01:1
DatepickerMinimalPerf.default	3625	3584	1.01:1
DropdownManyItemsPerf.default	391	387	1.01:1
MenuMinimalPerf.default	504	498	1.01:1
ProviderMinimalPerf.default	198	197	1.01:1
TooltipMinimalPerf.default	1297	1284	1.01:1
ChatMinimalPerf.default	429	430	1:1
DialogMinimalPerf.default	441	441	1:1
DividerMinimalPerf.default	204	203	1:1
DropdownMinimalPerf.default	1411	1416	1:1
ItemLayoutMinimalPerf.default	717	715	1:1
ListNestedPerf.default	328	328	1:1
PopupMinimalPerf.default	345	345	1:1
RadioGroupMinimalPerf.default	261	261	1:1
TableManyItemsPerf.default	1126	1127	1:1
ToolbarMinimalPerf.default	532	534	1:1
AccordionMinimalPerf.default	81	82	0.99:1
EmbedMinimalPerf.default	1853	1867	0.99:1
FlexMinimalPerf.default	155	156	0.99:1
ListMinimalPerf.default	311	314	0.99:1
MenuButtonMinimalPerf.default	949	961	0.99:1
CustomToolbarPrototype.default	1471	1486	0.99:1
TreeMinimalPerf.default	469	476	0.99:1
AlertMinimalPerf.default	155	158	0.98:1
AnimationMinimalPerf.default	295	301	0.98:1
LayoutMinimalPerf.default	194	197	0.98:1
ReactionMinimalPerf.default	205	209	0.98:1
SliderMinimalPerf.default	712	723	0.98:1
SplitButtonMinimalPerf.default	2231	2276	0.98:1
ChatWithPopoverPerf.default	183	189	0.97:1
ListCommonPerf.default	382	395	0.97:1
VideoMinimalPerf.default	426	437	0.97:1
InputMinimalPerf.default	519	541	0.96:1
TreeWith60ListItems.default	91	95	0.96:1
AttachmentMinimalPerf.default	77	81	0.95:1
BoxMinimalPerf.default	187	197	0.95:1
SegmentMinimalPerf.default	192	203	0.95:1
ButtonOverridesMissPerf.default	624	662	0.94:1
TextAreaMinimalPerf.default	271	292	0.93:1
ListWith60ListItems.default	350	381	0.92:1
ImageMinimalPerf.default	211	231	0.91:1

fabricteam · 2023-08-25T11:05:19Z

🕵 FluentUIV0 No visual regressions between this PR and main

fabricteam · 2023-08-25T11:07:47Z

🕵 fluentuiv9 No visual regressions between this PR and main

fabricteam · 2023-08-25T11:08:43Z

Perf Analysis (`@fluentui/react`)

No significant results to display.

All results

Scenario	Render type	Master Ticks	PR Ticks	Iterations
BaseButton	mount	635	640	5000
Breadcrumb	mount	1734	1739	1000
Checkbox	mount	1674	1724	5000
CheckboxBase	mount	1465	1523	5000
ChoiceGroup	mount	2995	2907	5000
ComboBox	mount	655	656	1000
CommandBar	mount	6314	6260	1000
ContextualMenu	mount	13943	13721	1000
DefaultButton	mount	754	763	5000
DetailsRow	mount	2169	2192	5000
DetailsRowFast	mount	2196	2197	5000
DetailsRowNoStyles	mount	2007	2037	5000
Dialog	mount	2692	2658	1000
DocumentCardTitle	mount	235	236	1000
Dropdown	mount	2001	2004	5000
FocusTrapZone	mount	1135	1145	5000
FocusZone	mount	1088	1065	5000
GroupedList	mount	41629	42146	2
GroupedList	virtual-rerender	20220	20015	2
GroupedList	virtual-rerender-with-unmount	50561	52507	2
GroupedListV2	mount	238	242	2
GroupedListV2	virtual-rerender	212	224	2
GroupedListV2	virtual-rerender-with-unmount	235	233	2
IconButton	mount	1114	1097	5000
Label	mount	343	353	5000
Layer	mount	2703	2721	5000
Link	mount	409	393	5000
MenuButton	mount	946	956	5000
MessageBar	mount	22035	22132	5000
Nav	mount	1968	1963	1000
OverflowSet	mount	794	787	5000
Panel	mount	1803	1795	1000
Persona	mount	727	745	1000
Pivot	mount	863	856	1000
PrimaryButton	mount	859	845	5000
Rating	mount	4538	4559	5000
SearchBox	mount	939	917	5000
Shimmer	mount	1868	1890	5000
Slider	mount	1310	1308	5000
SpinButton	mount	2862	2910	5000
Spinner	mount	402	383	5000
SplitButton	mount	1827	1838	5000
Stack	mount	413	418	5000
StackWithIntrinsicChildren	mount	839	848	5000
StackWithTextChildren	mount	2591	2563	5000
SwatchColorPicker	mount	6108	6038	5000
TagPicker	mount	1436	1511	5000
Text	mount	381	371	5000
TextField	mount	951	937	5000
ThemeProvider	mount	844	849	5000
ThemeProvider	virtual-rerender	588	585	5000
ThemeProvider	virtual-rerender-with-unmount	1273	1264	5000
Toggle	mount	612	617	5000
buttonNative	mount	192	196	5000

codesandbox-ci · 2023-08-28T09:07:31Z

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit 7221a0e:

Sandbox	Source
@fluentui/react 8 starter	Configuration
@fluentui/react-components 9 starter	Configuration

size-auditor · 2023-08-28T09:17:32Z

Asset size changes

Size Auditor did not detect a change in bundle size for any component!

Baseline commit: 42ed56cff2ef6c21fb2d5805d77c370dc3036ebe (build)

fabricteam · 2023-08-28T09:31:45Z

🕵 fluentuiv8 No visual regressions between this PR and main

YuanboXue-Amber · 2023-08-28T09:34:29Z

/azp run

azure-pipelines · 2023-08-28T09:34:48Z

Azure Pipelines successfully started running 4 pipeline(s).

packages/react-conformance/src/isConformant.ts

Hotell · 2023-09-21T12:23:42Z

packages/react-conformance/src/isConformant.ts

+ * Verifies that a test function has only one parameter. If so, this test does not require TS info.
+ */
+function isNonTypeTest<TProps = {}>(func: ConformanceTest<TProps>): func is BaseConformanceTest<TProps> {
+  return func.length === 1;


this is not the most robust or explicit API to have for such a critical functionality. looks like missed opportunity to design it properly while introducing breaking change. would be great to bring this to v-build sync to discuss things for better outcome. ty

YuanboXue-Amber added 2 commits August 25, 2023 12:34

disableTypeTests

f265899

chglog

5c27cf7

YuanboXue-Amber added 2 commits August 25, 2023 13:32

api

0406e18

fix other test

7bb77d9

YuanboXue-Amber changed the title ~~feat(react-conformance): new option disableTypeTests disables tests that requires type information~~ BREAKING(react-conformance): new option disableTypeTests disables tests that requires type information Aug 28, 2023

changelog

7b1efcd

YuanboXue-Amber added 2 commits August 28, 2023 11:19

type polish

b6aa69b

Merge branch 'master' into filter-conformance

9bdc6da

YuanboXue-Amber added 3 commits August 28, 2023 11:51

fix n*

f3b979b

type update

cb359c1

api

783467d

YuanboXue-Amber changed the title ~~BREAKING(react-conformance): new option disableTypeTests disables tests that requires type information~~ BREAKING(react-conformance): update test type and add new option disableTypeTests to disables tests that requires type information Aug 28, 2023

type+comment

76632b4

YuanboXue-Amber marked this pull request as ready for review August 28, 2023 11:10

YuanboXue-Amber requested review from a team as code owners August 28, 2023 11:10

tudorpopams self-requested a review August 29, 2023 12:22

ling1726 reviewed Sep 4, 2023

View reviewed changes

packages/react-conformance/src/isConformant.ts Show resolved Hide resolved

ling1726 approved these changes Sep 4, 2023

View reviewed changes

tudorpopams approved these changes Sep 6, 2023

View reviewed changes

Merge branch 'master' into filter-conformance

7221a0e

YuanboXue-Amber enabled auto-merge (squash) September 6, 2023 11:22

YuanboXue-Amber merged commit 8582884 into microsoft:master Sep 6, 2023

YuanboXue-Amber deleted the filter-conformance branch September 6, 2023 12:09

Hotell reviewed Sep 21, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BREAKING(react-conformance): update test type and add new option `disableTypeTests` to disables tests that requires type information #28988

BREAKING(react-conformance): update test type and add new option `disableTypeTests` to disables tests that requires type information #28988

YuanboXue-Amber commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

codesandbox-ci bot commented Aug 28, 2023 •

edited

Loading

size-auditor bot commented Aug 28, 2023 •

edited

Loading

fabricteam commented Aug 28, 2023 •

edited

Loading

YuanboXue-Amber commented Aug 28, 2023

azure-pipelines bot commented Aug 28, 2023

Hotell Sep 21, 2023

BREAKING(react-conformance): update test type and add new option disableTypeTests to disables tests that requires type information #28988

BREAKING(react-conformance): update test type and add new option disableTypeTests to disables tests that requires type information #28988

Conversation

YuanboXue-Amber commented Aug 25, 2023 • edited Loading

fabricteam commented Aug 25, 2023 • edited Loading

📊 Bundle size report

fabricteam commented Aug 25, 2023 • edited Loading

Perf Analysis (@fluentui/react-components)

fabricteam commented Aug 25, 2023 • edited Loading

Perf Analysis (@fluentui/react-northstar)

Potential regressions comparing to master

fabricteam commented Aug 25, 2023 • edited Loading

🕵 FluentUIV0 No visual regressions between this PR and main

fabricteam commented Aug 25, 2023 • edited Loading

🕵 fluentuiv9 No visual regressions between this PR and main

fabricteam commented Aug 25, 2023 • edited Loading

Perf Analysis (@fluentui/react)

codesandbox-ci bot commented Aug 28, 2023 • edited Loading

size-auditor bot commented Aug 28, 2023 • edited Loading

Asset size changes

fabricteam commented Aug 28, 2023 • edited Loading

🕵 fluentuiv8 No visual regressions between this PR and main

YuanboXue-Amber commented Aug 28, 2023

azure-pipelines bot commented Aug 28, 2023

Hotell Sep 21, 2023

Choose a reason for hiding this comment

BREAKING(react-conformance): update test type and add new option `disableTypeTests` to disables tests that requires type information #28988

BREAKING(react-conformance): update test type and add new option `disableTypeTests` to disables tests that requires type information #28988

YuanboXue-Amber commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

Perf Analysis (`@fluentui/react-components`)

fabricteam commented Aug 25, 2023 •

edited

Loading

Perf Analysis (`@fluentui/react-northstar`)

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

fabricteam commented Aug 25, 2023 •

edited

Loading

Perf Analysis (`@fluentui/react`)

codesandbox-ci bot commented Aug 28, 2023 •

edited

Loading

size-auditor bot commented Aug 28, 2023 •

edited

Loading

fabricteam commented Aug 28, 2023 •

edited

Loading