feat: Introduce HF Transformers to ModelBuilder by samruds · Pull Request #4368 · aws/sagemaker-python-sdk

samruds · 2024-01-10T22:08:36Z

Issue #, if available: #4284

Description of changes:

Introduce the HF Model Builder
Add unit tests

Testing done:
Tested changes using Local and Sagemaker endpoint mode

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

src/sagemaker/serve/builder/hf_dlc_builder.py

src/sagemaker/serve/model_server/hf_dlc/server.py

src/sagemaker/serve/builder/model_builder.py

src/sagemaker/serve/builder/hf_dlc_builder.py

src/sagemaker/serve/model_server/hf_dlc/server.py

src/sagemaker/serve/builder/hf_dlc_builder.py

src/sagemaker/serve/model_server/hf_dlc/server.py

src/sagemaker/serve/builder/hf_dlc_builder.py

gwang111 · 2024-01-11T19:08:57Z

please rename to feat: Introduce HF DLC to ModelBuilder

gwang111 · 2024-01-11T21:59:25Z

i think we should also add additional unit testing. I don't see coverage for the new hf_dlc_builder.py

samruds · 2024-01-16T20:21:54Z

i think we should also add additional unit testing. I don't see coverage for the new hf_dlc_builder.py

Added UT

src/sagemaker/serve/builder/hf_dlc_builder.py

src/sagemaker/serve/model_server/hf_dlc/prepare.py

src/sagemaker/serve/builder/hf_dlc_builder.py

mufaddal-rohawala · 2024-02-01T01:20:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7aaf6d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T01:48:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c7aaf6d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/serve/builder/transformers_builder.py

mohanasudhan · 2024-02-01T18:07:31Z

src/sagemaker/serve/model_server/multi_model_server/server.py

+        """Placeholder docstring"""
+        self.container = client.containers.run(
+            image,
+            "serve",


For more information : SageMaker overrides default CMD statements in a container by specifying the serve argument after the image name. The serve argument overrides arguments that you provide with the CMD command in the Dockerfile.

https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html#your-algorithms-inference-code-run-image

src/sagemaker/serve/model_server/multi_model_server/server.py

mufaddal-rohawala · 2024-02-01T19:41:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 3f778ad
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T19:43:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 3f778ad
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T19:50:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 3f778ad
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T21:20:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 617dc82
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T21:26:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 617dc82
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T21:59:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 617dc82
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T22:16:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 617dc82
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-01T22:18:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 617dc82
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T01:38:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a2ab955
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T01:57:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a06b406
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T02:14:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f8e820d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T02:25:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: faa93e5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T04:21:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5bd854e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T04:22:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5bd854e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T04:36:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 3d6636b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T05:45:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ba850c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T05:45:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ba850c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T05:48:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ba850c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T06:26:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ba850c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-02T06:46:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ba850c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

samruds requested a review from a team as a code owner January 10, 2024 22:08

samruds requested review from EvelynYan21 and removed request for a team January 10, 2024 22:08

gwang111 reviewed Jan 10, 2024

View reviewed changes

src/sagemaker/serve/builder/hf_dlc_builder.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 10, 2024

View reviewed changes

src/sagemaker/serve/model_server/hf_dlc/server.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 10, 2024

View reviewed changes

src/sagemaker/serve/builder/model_builder.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 10, 2024

View reviewed changes

src/sagemaker/serve/builder/hf_dlc_builder.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 10, 2024

View reviewed changes

src/sagemaker/serve/model_server/hf_dlc/server.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 10, 2024

View reviewed changes

src/sagemaker/serve/builder/hf_dlc_builder.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 11, 2024

View reviewed changes

src/sagemaker/serve/builder/hf_dlc_builder.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 11, 2024

View reviewed changes

src/sagemaker/serve/model_server/hf_dlc/server.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 11, 2024

View reviewed changes

src/sagemaker/serve/builder/hf_dlc_builder.py Outdated Show resolved Hide resolved

gwang111 reviewed Jan 11, 2024

View reviewed changes

src/sagemaker/serve/builder/hf_dlc_builder.py Outdated Show resolved Hide resolved

samruds force-pushed the master branch from ab1fff4 to ab48e1c Compare January 12, 2024 18:51

samruds closed this Jan 12, 2024

samruds force-pushed the master branch from 8079ac3 to f2b47ab Compare January 12, 2024 18:52

samruds changed the title ~~Introduce HF DLC to Model Builder~~ feat: Introduce HF DLC to ModelBuilder Jan 12, 2024

samruds reopened this Jan 12, 2024

samruds force-pushed the master branch from b5f4681 to e960f45 Compare January 16, 2024 20:16

jiapinw suggested changes Jan 16, 2024

View reviewed changes

samruds commented Jan 16, 2024

View reviewed changes

src/sagemaker/serve/builder/hf_dlc_builder.py Outdated Show resolved Hide resolved

samruds force-pushed the master branch 6 times, most recently from 7ee395a to 6b307c0 Compare January 17, 2024 20:39

mohanasudhan reviewed Feb 1, 2024

View reviewed changes

Merge branch 'master' into master

a81e0fa

samruds mentioned this pull request Feb 1, 2024

Remove duplicate code between MultiModelServer and TorchServer #4402

Open

Address PR feedbacks

3f778ad

mohanasudhan approved these changes Feb 1, 2024

View reviewed changes

Merge branch 'master' into master

617dc82

Fix test builds

ba850c2

benieric approved these changes Feb 2, 2024

View reviewed changes

Conversation

samruds commented Jan 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gwang111 commented Jan 11, 2024

Uh oh!

gwang111 commented Jan 11, 2024

Uh oh!

samruds commented Jan 16, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

Uh oh!

mohanasudhan Feb 1, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 1, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 2, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 2, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 2, 2024

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented Feb 2, 2024

AWS CodeBuild CI Report

samruds commented Jan 10, 2024 •

edited

Loading