[wasi+browser] bundle timezones into .wasm by pavelsavara · Pull Request #82250 · dotnet/runtime

pavelsavara · 2023-02-16T18:22:43Z

Motivation

the VFS is slow in the browser. I expect to save 50ms of startup time
this will bring de-duplication of files with same hash and bit smaller overall download size.
it will enable memory snapshot of the initialized runtime
necessary for WASI with <WasmSingleFileBundle>

Changes

TimeZoneInfo will use this new embedded TZ database for browser and wasi OS
- unless TZDIR env variable is set
- if TZDIR is set it would fallback to unix-like file system approach (as before).
added mono_wasm_get_bundled_file and mono_wasm_add_bundled_file
mono_wasm_add_bundled_file is called with file names from generated mono_wasm_register_bundle_timezones
added SystemNative_GetTimeZoneData which calls new mono_wasm_get_bundled_file
in of <WasmSingleFileBundle> we also generate mono_wasm_register_bundle_assemblies for WASI
removed dotnet.timezones.blat from runtime pack
added wasm-bundled-timezones.a to workloads
fixed TARGET_BROWSER detection for native/libs
do not set TZ env variable to UTC as that's not valid timezone anyway. dotnet will fallback to UTC.
create usr/share on browser VFS because it's mapped to SpecialFolder.CommonApplicationData which would not exist otherwise.
rewrote EmitWasmBundleObjectFile as new EmitWasmBundleFiles
- it can bundle any file not just assemblies
- it can generate code to to wasi clang input stream or to files for emcc
- it skips files which are already converted to
new Target GenerateTimezonesArchive in both wasi.proj and wasm.proj
- find all files to bundle, get hash for them
- pass the list to EmitWasmBundleFiles Task
  - [for wasi] object files will be produced
  - [for emcc] compile source files to obj file after this
- remove old source and object files
updated Target _GenerateAssemblyObjectFiles to use new EmitWasmBundleFiles task

future work

this will need SDK change for blazor
next PR make BlazorEnableTimeZoneSupport work again,
- when workload is available.
- via something like Add feature-switch for invariant TimeZone support #37973
make file bundling more incremental
similar bundling could be used for WASI and ICU data files

ghost · 2023-02-16T18:22:54Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

work in progress

Author:	pavelsavara
Assignees:	pavelsavara
Labels:	`arch-wasm`, `os-wasi`
Milestone:	8.0.0

src/native/libs/System.Native/entrypoints.c

pavelsavara · 2023-02-22T20:15:40Z

I will need to deal with BlazorEnableTimeZoneSupport
See SDK and #37973

…nix.NonAndroid.cs Co-authored-by: Ankit Jain <radical@gmail.com>

…fferent asset types

pavelsavara · 2023-02-28T11:04:50Z

@radical do you have further feedback ?

src/libraries/System.Private.CoreLib/src/System/TimeZoneInfo.Unix.NonAndroid.cs

src/tasks/WasmAppBuilder/EmitWasmBundleBase.cs

.

Co-authored-by: Ankit Jain <radical@gmail.com>

radical

LGTM 👍

radical · 2023-03-01T08:19:29Z

src/mono/wasm/wasm.proj

+    <RunWithEmSdkEnv Command="$(EmccCmd) -xc -c @$(_WasmTimezonesSourcesRsp)"
+          WorkingDirectory="$(WasmObjDir)"
+          EmSdkPath="$(EMSDK_PATH)"
+          ConsoleToMsBuild="true"
+          IgnoreStandardErrorWarningFormat="true">
+    </RunWithEmSdkEnv>


For a follow up PR, consider using EmccCompile task which compiles in parallel.

pavelsavara · 2023-03-01T11:54:36Z

/azp run runtime-wasm

azure-pipelines · 2023-03-01T11:54:53Z

Azure Pipelines successfully started running 1 pipeline(s).

ilonatommy

Looks good, I especially like the description ✨

src/mono/mono/utils/mono-dl-wasm.c

pavelsavara · 2023-03-01T15:08:51Z

CI failures are unrelated.

pavelsavara · 2023-03-03T13:47:09Z

Original size of dotnet.timezones.blat was 336270 bytes.
The size of new data embedded in the dotnet.wasm is 221991 bytes.

Some of the startup speed improvement is also caused by it.

pavelsavara added arch-wasm WebAssembly architecture os-wasi Related to WASI variant of arch-wasm labels Feb 16, 2023

pavelsavara added this to the 8.0.0 milestone Feb 16, 2023

pavelsavara self-assigned this Feb 16, 2023

ghost added the area-Build-mono label Feb 16, 2023

This was referenced Feb 16, 2023

OSX infra issue - prereq check for 'pkg-config' missing #82240

Closed

Roslyn source generator crash on mono/linux/arm64 #81123

Closed

SupportedLinuxPlatforms_IsSupportedIsTrue fails on distros without quic #82077

Closed

jakobbotsch mentioned this pull request Feb 17, 2023

[QUIC] Missing libmsquic on tested platforms #81901

Closed

4 tasks

pavelsavara force-pushed the wasi_bundle_timezones branch 3 times, most recently from 18e709c to a45ecbf Compare February 21, 2023 17:00

pavelsavara changed the title ~~[wasi] bundle timezones into .wasm~~ [wasi+browser] bundle timezones into .wasm Feb 21, 2023

pavelsavara commented Feb 21, 2023

View reviewed changes

src/native/libs/System.Native/entrypoints.c Outdated Show resolved Hide resolved

build-analysis bot mentioned this pull request Feb 21, 2023

Tracking issue for CI build timeouts #76454

Closed

pavelsavara force-pushed the wasi_bundle_timezones branch from 9c2f6d6 to 3d4cfb7 Compare February 22, 2023 10:34

wip

89d4f74

pavelsavara force-pushed the wasi_bundle_timezones branch from 37a1350 to 89d4f74 Compare February 22, 2023 17:47

Merge branch 'main' into wasi_bundle_timezones

c0e9a8b

pavelsavara requested a review from mkhamoyan February 23, 2023 15:42

pavelsavara marked this pull request as ready for review February 23, 2023 15:52

pavelsavara requested review from kg, lambdageek, lewing, marek-safar, radical and vargaz as code owners February 23, 2023 15:52

fix

42db50f

pavelsavara and others added 5 commits February 27, 2023 09:25

Update src/libraries/System.Private.CoreLib/src/System/TimeZoneInfo.U…

ac2521d

…nix.NonAndroid.cs Co-authored-by: Ankit Jain <radical@gmail.com>

feedback

8cbf72f

added /usr/share/zoneinfo/ prefix - namespace so that we don't mix di…

01d3c60

…fferent asset types

Merge branch 'main' into wasi_bundle_timezones

9d3e75c

fix

26d5f31

pavelsavara requested a review from radical February 27, 2023 15:50

pavelsavara mentioned this pull request Feb 28, 2023

[wasi][icu] Add ICU sharding for wasi #82614

Open

radical reviewed Mar 1, 2023

View reviewed changes

pavelsavara and others added 2 commits March 1, 2023 08:33

Update src/tasks/WasmAppBuilder/EmitWasmBundleBase.cs

c8af4b8

Co-authored-by: Ankit Jain <radical@gmail.com>

feedback

e674fd7

radical approved these changes Mar 1, 2023

View reviewed changes

radical reviewed Mar 1, 2023

View reviewed changes

maraf approved these changes Mar 1, 2023

View reviewed changes

ilonatommy approved these changes Mar 1, 2023

View reviewed changes

src/mono/mono/utils/mono-dl-wasm.c Show resolved Hide resolved

This was referenced Mar 1, 2023

Test failure: System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests/CreateChain_Hybrid #25979

Closed

System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice failure #65448

Closed

pavelsavara merged commit 340508f into dotnet:main Mar 1, 2023

This was referenced Mar 1, 2023

System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice failure #82837

Open

Some System.Buffers.Binary.Tests.ReverseEndiannessUnitTests tests broken on main #82868

Closed

mkhamoyan mentioned this pull request Mar 2, 2023

[wasi][icu] bundle icu files into .wasm #82895

Closed

pavelsavara mentioned this pull request Mar 3, 2023

Add feature-switch for invariant TimeZone support #37973

Closed

ilonatommy mentioned this pull request Mar 28, 2023

Update off-by-default analyzers to account for rename dotnet/sdk#31460

Merged

ghost locked as resolved and limited conversation to collaborators Apr 2, 2023

pavelsavara deleted the wasi_bundle_timezones branch September 2, 2024 15:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wasi+browser] bundle timezones into .wasm#82250

[wasi+browser] bundle timezones into .wasm#82250
pavelsavara merged 26 commits intodotnet:mainfrom
pavelsavara:wasi_bundle_timezones

pavelsavara commented Feb 16, 2023 •

edited

Loading

Uh oh!

ghost commented Feb 16, 2023

Uh oh!

Uh oh!

pavelsavara commented Feb 22, 2023 •

edited

Loading

Uh oh!

pavelsavara commented Feb 28, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

radical left a comment

Uh oh!

radical Mar 1, 2023

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

azure-pipelines bot commented Mar 1, 2023

Uh oh!

ilonatommy left a comment

Uh oh!

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

pavelsavara commented Mar 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

pavelsavara commented Feb 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

future work

Uh oh!

ghost commented Feb 16, 2023

Uh oh!

Uh oh!

pavelsavara commented Feb 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pavelsavara commented Feb 28, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

radical left a comment

Choose a reason for hiding this comment

Uh oh!

radical Mar 1, 2023

Choose a reason for hiding this comment

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

azure-pipelines bot commented Mar 1, 2023

Uh oh!

ilonatommy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

pavelsavara commented Mar 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pavelsavara commented Feb 16, 2023 •

edited

Loading

pavelsavara commented Feb 22, 2023 •

edited

Loading