Skip to content

chore: cleanup & improve networking#1433

Merged
0utplay merged 76 commits intonightly-jdk22from
network-refactor
Jul 7, 2024
Merged

chore: cleanup & improve networking#1433
0utplay merged 76 commits intonightly-jdk22from
network-refactor

Conversation

@derklaro
Copy link
Member

@derklaro derklaro commented Jul 2, 2024

Motivation

The current network implementation has some issues that needed to be addressed. Most noticeble are:

  • issues with very high native memory consumption due to netty not releasing native memory instantly
  • issues with a lot of loaded classes that were auto generated
  • use of ASM which leads to issues with server implementations
  • rpc code generation can be noticeably improved, especially to ease the usage
  • and many places where the old code was just messy and could be replaced with a cleaner & easier to read version

Modification

  • bump the required java version for cloudnet to 22 & enable preview features.
  • replace the usages of ASM in code generation with the java classfile api
  • rewrite rpc handling & code generation completely
  • improve default values for settings (for example lower thread counts for packet handling)
  • improve chunked data transfer, prevent race conditions when querying files
  • use custom buffer allocator that releases native memory immediately instead of relying on GC
  • higher test coverage
  • ... and many more small cleanups and improvements

Result

Improved networking that should ease the use for external users and definitely decrease the amount of reported errors.

derklaro and others added 30 commits May 30, 2024 19:05
@derklaro
begin some work on refactoring network code
1b9c7be
@derklaro
some more work to improve networking (RPC specifically)
b4d9728
@derklaro
improve object mapper related things
5b7368e
@derklaro
Merge branch 'nightly' of github.com:CloudNetService/CloudNet-v3 into…
516fc45 
… nightly
@derklaro
finish most of the new rpc implementation (except api impl generation)
cd41525
@derklaro
improve support for async methods
08175d5
@derklaro
PatternObjectSerializer: include flags
a6407c9
@derklaro
begin work on new api implementation generator
781a7ec
@derklaro
be more strict which classes will be implemented with code generation
45b319b
@derklaro
small logic fixup
8b02e08
@derklaro
replace some magic array offsets
b9c6c98
@derklaro
more documentation & very small logic improvements
53b4688
@derklaro
use core constant for MethodTypeDesc class descriptor
e44f083
@derklaro
work on chain method implementation generation
6ce20c5
@derklaro
add ability to inject some rpc-specific args into a super constructor
1938b4c
@derklaro
implement implementation builder
0f61b61
@derklaro
drop rpc validation annotation
c01703c
@derklaro
merge nightly
0666e05
@derklaro
add remaining docs, remove unused classes
1209ed6
@derklaro
small refactors in chunked data transfer
e26147a
@derklaro
small progress
3675925
@derklaro
final refactor touches (for now)
37633dc
@derklaro
remove unused AsmHelper
e6586fc
@derklaro
fix ssl configuration handling for services
91c1b64
@derklaro
remove usages of RPCValidation
22b977c
@derklaro
fix compilation issues, resolve some style mismatches
7244e61
@derklaro
remove all usages of -XX:+DisableExplicitGC
9868f30
@derklaro
start fixing tests, fix first small found generation issues
19aa0fb
@derklaro
revert unnecessary/bad auto cleanup of IJ
72bdbc2
@derklaro
re-add rpc handler test, fix some more small issues
c779035
derklaro and others added 11 commits June 25, 2024 22:42
@derklaro derklaro added v: 4.X This pull should be included in the 4.0 release in: driver An issue/pull request releated to the driver module code t: fix A pull request introducing a fix for a bug. t: improvement The pull request improves existing code a: api An issue/pull request that is related to api changes. labels Jul 2, 2024
@derklaro derklaro added this to the 4.0.0-RC11 milestone Jul 2, 2024
@derklaro derklaro requested a review from 0utplay July 2, 2024 12:39
@derklaro derklaro self-assigned this Jul 2, 2024
@derklaro derklaro changed the title Network refactor chore: cleanup & improve networking Jul 2, 2024
@github-actions
Copy link

github-actions bot commented Jul 2, 2024 •
edited
Loading

Test Results

 47 files  +3   47 suites  +3   2m 13s ⏱️ +52s
420 tests +6  420 ✅ +6  0 💤 ±0  0 ❌ ±0 
751 runs  +6  751 ✅ +6  0 💤 ±0  0 ❌ ±0 

Results for commit 4583b29. ± Comparison against base commit af2222d.

♻️ This comment has been updated with latest results.

0utplay
0utplay approved these changes Jul 6, 2024
View reviewed changes
Copy link
Member

@0utplay 0utplay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just small comments and some questions

@0utplay 0utplay merged commit 22abdbe into nightly-jdk22 Jul 7, 2024
@0utplay 0utplay deleted the network-refactor branch July 7, 2024 13:43
derklaro added a commit that referenced this pull request Jul 8, 2024
@derklaro
chore: cleanup & improve networking (#1433)
4b82fa9 
The current network implementation has some issues that needed to be
addressed. Most noticeble are:
- issues with very high native memory consumption due to netty not
releasing native memory instantly
- issues with a lot of loaded classes that were auto generated
- use of ASM which leads to issues with server implementations
- rpc code generation can be noticeably improved, especially to ease the
usage
- and many places where the old code was just messy and could be
replaced with a cleaner & easier to read version

- bump the required java version for cloudnet to 22 & enable preview
features.
- replace the usages of ASM in code generation with the java classfile
api
- rewrite rpc handling & code generation completely
- improve default values for settings (for example lower thread counts
for packet handling)
- improve chunked data transfer, prevent race conditions when querying
files
- use custom buffer allocator that releases native memory immediately
instead of relying on GC
- higher test coverage
- ... and many more small cleanups and improvements

Improved networking that should ease the use for external users and
definitely decrease the amount of reported errors.
derklaro added a commit that referenced this pull request Jul 9, 2024
@derklaro
chore: cleanup & improve networking (#1433)
e4999d6 
The current network implementation has some issues that needed to be
addressed. Most noticeble are:
- issues with very high native memory consumption due to netty not
releasing native memory instantly
- issues with a lot of loaded classes that were auto generated
- use of ASM which leads to issues with server implementations
- rpc code generation can be noticeably improved, especially to ease the
usage
- and many places where the old code was just messy and could be
replaced with a cleaner & easier to read version

- bump the required java version for cloudnet to 22 & enable preview
features.
- replace the usages of ASM in code generation with the java classfile
api
- rewrite rpc handling & code generation completely
- improve default values for settings (for example lower thread counts
for packet handling)
- improve chunked data transfer, prevent race conditions when querying
files
- use custom buffer allocator that releases native memory immediately
instead of relying on GC
- higher test coverage
- ... and many more small cleanups and improvements

Improved networking that should ease the use for external users and
definitely decrease the amount of reported errors.
derklaro added a commit that referenced this pull request Jul 12, 2024
@derklaro
chore: cleanup & improve networking (#1433)
ab8903e 
The current network implementation has some issues that needed to be
addressed. Most noticeble are:
- issues with very high native memory consumption due to netty not
releasing native memory instantly
- issues with a lot of loaded classes that were auto generated
- use of ASM which leads to issues with server implementations
- rpc code generation can be noticeably improved, especially to ease the
usage
- and many places where the old code was just messy and could be
replaced with a cleaner & easier to read version

- bump the required java version for cloudnet to 22 & enable preview
features.
- replace the usages of ASM in code generation with the java classfile
api
- rewrite rpc handling & code generation completely
- improve default values for settings (for example lower thread counts
for packet handling)
- improve chunked data transfer, prevent race conditions when querying
files
- use custom buffer allocator that releases native memory immediately
instead of relying on GC
- higher test coverage
- ... and many more small cleanups and improvements

Improved networking that should ease the use for external users and
definitely decrease the amount of reported errors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@0utplay 0utplay 0utplay approved these changes

Assignees

@derklaro derklaro

Labels

a: api An issue/pull request that is related to api changes. in: driver An issue/pull request releated to the driver module code t: fix A pull request introducing a fix for a bug. t: improvement The pull request improves existing code v: 4.X This pull should be included in the 4.0 release

Projects

None yet

Milestone

4.0.0-RC11

Development

Successfully merging this pull request may close these issues.

2 participants

@derklaro @0utplay