Skip to content

Fix unicode path issues#466

Merged
yao-msft merged 3 commits intomicrosoft:masterfrom
yao-msft:fixunicodepath
Jun 29, 2020
Merged

Fix unicode path issues#466
yao-msft merged 3 commits intomicrosoft:masterfrom
yao-msft:fixunicodepath

Conversation

@yao-msft
Copy link
Contributor

@yao-msft yao-msft commented Jun 27, 2020

For #181 #315

Cause

There're multiple places a std::filesystem::path is created with narrow string input. It's ok most of the time since we control the expected path(i.e. index.msix, etc). But if the file path involves user input, this will be a problem.

Change

Examine the code and determine places that a path involves user input. Call Utility::ConvertToUtf16 before creating std::filesystem::path.

Validation

Updated E2E tests to contain extended character scenario. Install e2e tests will be updated when it's fully enabled.
Manually tested install scenario by creating a user account with extended characters and try some installation.

Microsoft Reviewers: Open in CodeFlow

@yao-msft yao-msft requested a review from a team as a code owner June 27, 2020 02:42
SHELLEXECUTEINFOA execInfo = { 0 };
execInfo.cbSize = sizeof(SHELLEXECUTEINFO);
SHELLEXECUTEINFOW execInfo = { 0 };
execInfo.cbSize = sizeof(SHELLEXECUTEINFOW);
Copy link
Member

@JohnMcPMS JohnMcPMS Jun 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHELLEXECUTEINFOW [](start = 37, length = 17)

Just use sizeof(execInfo), then you don't have to chase its type. #Closed

execInfo.lpFile = filePathUTF8Str.c_str();
execInfo.lpParameters = args.c_str();
execInfo.lpFile = filePath.c_str();
execInfo.lpParameters = Utility::ConvertToUTF16(args).c_str();
Copy link
Member

@JohnMcPMS JohnMcPMS Jun 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Utility::ConvertToUTF16(args) [](start = 36, length = 29)

This is a temporary whose life ends at the semicolon. You need to keep it local to be able to pass it in this way. #Closed

@JohnMcPMS
Copy link
Member

JohnMcPMS commented Jun 27, 2020

This seems like a lot of band-aids that won't cover new cases well. Is there a more fundamental change that we can make to prevent future recurrence? If we had char8_t, that would solve the majority of the problems, but we don't with C++17. We could turn it on, either with /Zc:char8_t or /std:c++latest, but they both have their issues. However, it is the best bet to prevent all future issues.

But changing every std::string to std::u8string (or at least every one that might possibly be UTF-8), and the knock on effects that would have, could be a massive change.

The only thought for an intermediate solution would be to create a new filesystem type that treats std::string as std::u8string, much like I did for NormalizedString. But even that seems like it would be easy to miss one of the functions that we don't use or don't notice.

Can you think of any other ways to prevent potential future regression in this area? #Resolved

Copy link
Member

@JohnMcPMS JohnMcPMS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🕐

@ghost ghost added the Needs-Author-Feedback Issue needs attention from issue or PR author label Jun 27, 2020
@yao-msft
Copy link
Contributor Author

Discussed offline of a couple options. All require fair amount of work over a bug fix. Will file an issue to track the larger work.


In reply to: 650522963 [](ancestors = 650522963)

@ghost ghost removed the Needs-Author-Feedback Issue needs attention from issue or PR author label Jun 29, 2020
@yao-msft
Copy link
Contributor Author

yao-msft commented Jun 29, 2020

Discussed offline of a couple options. All require fair amount of work over a bug fix. Will file an issue to track the larger work.

In reply to: 650522963 [](ancestors = 650522963)

#473 filed #Resolved

Copy link
Member

@JohnMcPMS JohnMcPMS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@yao-msft yao-msft merged commit 242fa15 into microsoft:master Jun 29, 2020
@yao-msft yao-msft deleted the fixunicodepath branch June 29, 2020 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants