v0.10.1

What's Changed

• chore(deps-dev): bump brotli from 1.1.0 to 1.2.0 in /python by @dependabot[bot] in #751
• chore(deps-dev): bump urllib3 from 2.5.0 to 2.6.0 in /python by @dependabot[bot] in #756
• add more LDBC benchmark datasets by @james-willis in #764
• feat(pregel): add requiredSrcColumns/requiredDstColumns API for triplet memory optimization by @joelrobin18 in #763
• fix: mixing directed and undirected edges in motif throws parse error by @goungoun in #754
• Fix: Throw IllegalArgumentException for null IDs/Src/Dst in toGraphX (Fixes #765) by @JeganVG in #766
• chore(deps-dev): bump urllib3 from 2.6.0 to 2.6.3 in /python by @dependabot[bot] in #767
• Update README.md to fix broken network motif tutorial link by @russelljurney-upside in #770
• revert ConnectedComponents back to 0.9.3 version to fix perf regression by @james-willis in #772

New Contributors

• @JeganVG made their first contribution in #766
• @russelljurney-upside made their first contribution in #770

Full Changelog: v0.10.0...v0.10.1

v0.10.0

What's Changed

• feat: initial commit with a copy of GraphX code by @SemyonSinchenko in #680
• chore: update image paths in README files to new directory location by @SemyonSinchenko in #683
• feat: performance improvement in GraphX CDLP by @SemyonSinchenko in #681
• feat: fix memory leaks in GraphX by @SemyonSinchenko in #687
• feat: improvements in GF-based algorithms performance by @SemyonSinchenko in #690
• feat: add LDBC Connected Components example by @SemyonSinchenko in #692
• hotfix: java 8 did not have Path.of by @SemyonSinchenko in #694
• feat: variable length pattern matching by @goungoun in #685
• doc: update docs by @SemyonSinchenko in #699
• feat: performance and API sync by @SemyonSinchenko in #698
• feat: add validate method by @SemyonSinchenko in #695
• feat: add Rocha–Thatte cycle detection algorithm by @SemyonSinchenko in #700
• feat: aggregatemessages multiple columns by @SemyonSinchenko in #701
• chore: update Dockerfile by @SemyonSinchenko in #702
• feat: change default branch to "main" by @SemyonSinchenko in #703
• feat: add a test for pagerank by @SemyonSinchenko in #707
• feat: method to transform graph to undirected by @SemyonSinchenko in #705
• feat: simplified implementation of triplets by @SemyonSinchenko in #708
• feat: better TriangleCount by @SemyonSinchenko in #704
• doc: Documentation on setting up the environment by @joelrobin18 in #719
• chore: Unpersist cached data from the test suite by @joelrobin18 in #720
• feat: refactoring of python && docs && sync pyspark with core by @SemyonSinchenko in #711
• feat: add bidirectional motif pattern by @goungoun in #717
• feat: add undirected edge motif pattern by @goungoun in #712
• Improve result formatting for variable length pattern and undirected pattern by @goungoun in #728
• bug: prevent negated bidirectional edge by @goungoun in #739
• Add Graphframe Type Degree Methods by @joelrobin18 in #742
• fix: make py4j pyspark api robust by @SemyonSinchenko in #743
• feat: K-Core by @SemyonSinchenko in #740
• feat: reduce memory consumption of cycles detection by @SemyonSinchenko in #731
• feat: handle edges directions in SP by @SemyonSinchenko in #737
• feat: maximal independent set by @SemyonSinchenko in #744
• feat: prepare the release 0.10.0 by @SemyonSinchenko in #725

New Contributors

• @goungoun made their first contribution in #685
• @joelrobin18 made their first contribution in #719

Full Changelog: v0.9.3...v0.10.0

v0.9.3

What's Changed

• bug: Don't publish root project by @Kimahriman in #661
• feat: add tests for the case from #375 by @SemyonSinchenko in #663
• feat: localCheckpoints + docs by @SemyonSinchenko in #662
• feat: replace UDF by built-in function in skewedJoin by @SemyonSinchenko in #666
• feat: initial top-level structure of PropertyGraph by @SemyonSinchenko in #613
• fix: update javadocs to fix the build by @SemyonSinchenko in #670
• feat: S-sized benchmarks by @SemyonSinchenko in #671
• feat: updates in SBT and new site engine by @SemyonSinchenko in #673
• hotfix: fix broken links in docs by @SemyonSinchenko in #675
• feat: prepare 0.9.3 release by @SemyonSinchenko in #677

Full Changelog: v0.9.2...v0.9.3

v0.9.2

Overview

• fix publishing issues
• fix docs publishing issues

What's Changed

• fix: publish connect sources and javadocs by @SemyonSinchenko in #654
• feat: fix docs path by @SemyonSinchenko in #656
• feat: 0.9.2 release by @SemyonSinchenko in #658

Full Changelog: v0.9.1...v0.9.2

GraphFrames v0.9.1

Overview

Mostly this release is about publishing fixes (version string, README, PySpark Classic dependencies, etc.). There are no changes in core or API.

What's Changed

• Fix versioning by @Kimahriman in #641
• bug: Let classic python version work without grpc deps by @Kimahriman in #647
• fix: resolve gf-connect is not published by @SemyonSinchenko in #645
• feat: prepare the 0.9.1 release by @SemyonSinchenko in #650

Full Changelog: v0.9.0...v0.9.1

GraphFrames v0.9.0

Overview

• Spark Connect support
• Spark 4.x support
• Performance improvements in CC
• Updated API for Pregel
• GraphX-free impl of CDLP and SP
• New groupId io.graphframes
• New PyPi ID: graphframes-py
• A lot of additional changes and fixes
• Updated documentation

What's Changed

• Add JVM args to fix sbt.ForkMain error by @EnricoMi in #438
• Add Spark 3.5.0 support by @EnricoMi in #436
• Add log4j2.properties to test resources by @EnricoMi in #437
• Fix connected component by @WeichenXu123 in #454
• Update doc and releasing guide by @WeichenXu123 in #455
• Update quick-start.md to use latest build of GraphFrames jar by @rjurney in #461
• Java opt MaxPermSize --> MaxMetaspaceSize, includes for Java 17+ sun.nio.ch, java.lang ALL-UNNAMED by @rjurney in #464
• Upgrade github actions by @bjornjorgensen in #471
• Update to spark-version: 3.5.4 and fix Connected component suite by @bjornjorgensen in #478
• Add Type hints for Python code by @bjornjorgensen in #480
• Fix the bug I introduced previously when adapting build.sbt for JDK 11 by @rjurney in #476
• Update scala code by @bjornjorgensen in #479
• Use vertices SparkSession by @Nassizouz in #474
• Update .mailmap to include me by @rjurney in #482
• Update .mailmap to include rjurney by @rjurney in #483
• [DOCS] Adding new super cool logo to the docs by @SauronShepherd in #495
• adding favicon by @SauronShepherd in #497
• chore: drop sbt-spark-packages && update plugins by @SemyonSinchenko in #487
• Add Pull request template by @bjornjorgensen in #472
• doc: update badges in README by @SemyonSinchenko in #509
• chore: add scalafmt and re-format all the code by @SemyonSinchenko in #507
• 1 of 3: Build a graphframes Python package during the build process by @rjurney in #512
• Rjurney/motif tutorial code min by @rjurney in #520
• 3 of 3: Documentation cleanup and update. Added a motif finding tutorial. by @rjurney in #511
• chore: replace setup-scala action by @SemyonSinchenko in #522
• chore: suppress warnings about incomplete patterns by @SemyonSinchenko in #526
• chore: testing against both java-11 and java-17 by @SemyonSinchenko in #525
• chore: force python code style check in CI by @SemyonSinchenko in #527
• chore: pre-commit hook by @SemyonSinchenko in #534
• Convert nose / unittest tests to pytest by @rjurney in #524
• fixed a minor nit usage issue by @dmatrix in #535
• feat: power iteration clustering by @SemyonSinchenko in #529
• Add a missing image in preparation for docs release by @rjurney in #537
• feat: SparkConnect support by @SemyonSinchenko in #506
• [ConnectedComponents] Memory leak with unpersisted DataFrames in the last round by @SauronShepherd in #552
• feat: add early stopping to Pregel by @SemyonSinchenko in #550
• Changes the output label of CC to min of original ID by @architch in #320
• Quote column names with backticks by @EnricoMi in #370
• feat: add an experimental implementation of SP in GF by @SemyonSinchenko in #558
• feat: add LDBC tests by @SemyonSinchenko in #570
• chore: add "-Xlint" and scalafix checks && fix all errors && update pre-commit by @SemyonSinchenko in #575
• chore: automate docs publishing by @SemyonSinchenko in #566
• feat: add vertices "voting" ability and optional flags to Pregel by @SemyonSinchenko in #581
• feat: python publish by @SemyonSinchenko in #577
• feat: add a new mixin WithMaxIter and propogate it to GraphX CC by @SemyonSinchenko in #590
• Allow setting default ConnectedComponents configurations from spark config by @james-willis in #563
• Bump setuptools from 78.1.0 to 78.1.1 in /python by @dependabot[bot] in #596
• Removing unused file as per #585 by @Conor0Callaghan in #592
• feat: LabelPropagation without GraphX by @SemyonSinchenko in #587
• feat: sbt-ci-release by @SemyonSinchenko in #598
• Bump requests from 2.32.3 to 2.32.4 in /python by @dependabot[bot] in #601
• Bump protobuf from 5.29.4 to 5.29.5 in /python by @dependabot[bot] in #606
• Bump urllib3 from 2.4.0 to 2.5.0 in /python by @dependabot[bot] in #607
• feat: Scala publish CI GitHub Action by @SemyonSinchenko in #600
• feat: Spark 4.0 support by @Kimahriman in #608
• chore: Add issue templates by @SemyonSinchenko in #605
• chore: Cleanup assembly and shading by @Kimahriman in #617
• chore: fix shading problems and slightly update CI by @SemyonSinchenko in #615
• fix: try to fix publishing by @SemyonSinchenko in #619
• chore: add useLabelsAsComponents configuration for connected components by @SemyonSinchenko in #623
• doc: update README with details on nightly builds by @SemyonSinchenko in #622
• README and .gitignore update to add GraphFrames classes by @rjurney in #629
• allow useLabelsAsComponents to be set locally on the ConnectedComponents instance by @james-willis in #632
• feat: set plugin and update pyproject by @SemyonSinchenko in #634
• feat: planning release 0.9.0 by @SemyonSinchenko in #626
• fix: remove version.sbt by @SemyonSinchenko in #637

New Contributors

• @bjornjorgensen made their first contribution in #471
• @Nassizouz made their first contribution in #474
• @SauronShepherd made their first contribution in #495
• @SemyonSinchenko made their first contribution in #487
• @dmatrix made their first contribution in #535
• @architch made their first contribution in #320
• @james-willis made their first contribution in #563
• @dependabot[bot] made their first contribution in #596
• @Conor0Callaghan made their first contribution in #592
• @Kimahriman made their first contribution in #608

Full Changelog: v0.8.3...v0.9.0

GraphFrames 0.8.3

Major Changes

• Support Spark 3.3 / Scala 2.12 , Spark 3.4 / Scala 2.12 and Scala 2.13, Spark 3.5 / Scala 2.12 and Scala 2.13
• Remove deprecated SQLContext from code

What's Changed

• Use official Spark 3.2 PyPI release in CI
• Upgrade to latest spark versions 3.3.0, 3.2.2 and 3.1.3
• Python: Replace SQLContext with SparkSession
• Update CI actions and dependencies
• Add support for scala 2.13 (
• Remove deprecated sqlContext in graphframe scala code
• Spark 3.4.1 and drop EOL Spark versions 3.0 and 3.1

New Contributors

• eejbyfeldt
• ttc0419

Full Changelog: v0.8.2...v0.8.3

GraphFrames 0.8.2

Major Changes

• Support Spark 3.1 / Scala 2.12 and Spark 3.2 / Scala 2.12.

What's Changed

• [ML-12428] Upgrade CI spark version to 3.0.1 by @WeichenXu123 in #372
• Extend to Spark 3.1 by @EnricoMi in #381
• Correct the order of the analogy by @johnnyleitrim in #377
• Replace bintray by @menglewis in #385
• Switch from Travis CI to GitHub Actions by @mengxr in #402
• Build against Spark 3.2 by @mengxr in #400

New Contributors

• @EnricoMi made their first contribution in #381
• @johnnyleitrim made their first contribution in #377
• @menglewis made their first contribution in #385

Full Changelog: v0.8.1...v0.8.2

GraphFrames 0.8.1

• Build against official Spark 3.0.0 release.
• Cross build: Spark 3.0/Scala 2.12, Spark 2.4/Scala 2.11, Spark 2.4/Scala 2.12 (Experimental).
• Fixed an error in shortest path (#367).
• Removed the use of deprecated methods (#363).

GraphFrames 0.8.0

• Support Spark 2.4 / Scala 2.11 and Spark 3.0 (Preview) / Scala 2.12.