Skip to content

Internals re-implementation#1339

Merged
lorenzo merged 24 commits intomasterfrom
refactor-internals
May 6, 2018
Merged

Internals re-implementation#1339
lorenzo merged 24 commits intomasterfrom
refactor-internals

Conversation

@lorenzo
Copy link
Member

@lorenzo lorenzo commented Apr 28, 2018

Adding new features to Phinx was proving difficult for myself, due to its current arquitechture.

I decided to re-implement most of the internals as an attempt to remedy the situation and open up a way to add several more improvements in the future. I'll start with the benefits I was able to extract for this new architecture:

Benefits so far

  • Migrations run significantly faster: Instructions are bundled into as little ALTER instructions as it can do
  • Separation of fast VS slow instructions: Things like adding indexes and creating foreign keys are differed to be run last. This opens the way for executing the instructions concurrently in the database for even faster migrations.

The new architecture

Running migrations is very similar to compiling and executing a program. The executing a php file, for example, the parser will transform the instruction into an intermediate structure and then it executes it as it reads it.Optionally, php can also re-order some parts of the intermediate structure to add some optimisations.

This new architecture fits the same idea:

  • The migration files are the programming language
  • Phinx is the compiler
  • The adapters are the machines running the program

The problem with the old architecture was that instructions were sent immediately to the machine, instead of creating the intermediate structure. This new architecture builds an Intent, which is then transformed and optimised in several steps before it is finally sent to the database.

The new program flow is in broad terms as follows:

  • Migration files internally create an Intent object containing all Action objects to execute
  • Action objects are just value objects representing something to do in the database
  • A Plan object is used to transform the Intent into a set of intermediate structures: Either NewTable or AlterTable objects
  • The plan steps are then delegated to the adapters, which once again transform the AlterTable objects into another object called AlterInstructions
  • AlterInstructions are then compiled down to SQL and executed one by one in the database.

As you may guess, each transformation opens the possibility of adding certain optimisations. During this refactoring I only added one of such optimisations: The ability to run all ALTER instructions for a table as a single command. This is specially beneficial for Mysql and Postgresql

Breaking changes

This is a very significant breaking change, and I guess we can have that luxury as phinx has not been marked as stable. From the user point of view, there should be no visible changes, but for anyone extending the internals, this will break their code for sure.

lorenzo added 8 commits April 8, 2018 12:42
Using abumdant value objects to implement an interpreter-like machine
with the name "Plan".

The idea is that the plan can group the actions in a more intelligent
way, that is also faster for the database to execute in most cases.

Also refactored so that effects are treated more as data, this helps us
compose alter statamentes into a single call when it is an option.
Implemented again rversible migrations
@stickler-ci

This comment has been minimized.

@cakephp cakephp deleted a comment from stickler-ci Apr 28, 2018
@cakephp cakephp deleted a comment from stickler-ci Apr 28, 2018
@codecov
Copy link

codecov bot commented Apr 28, 2018

Codecov Report

Merging #1339 into master will decrease coverage by 1.03%.
The diff coverage is 79.02%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1339      +/-   ##
============================================
- Coverage     75.48%   74.44%   -1.04%     
- Complexity        0     1771    +1771     
============================================
  Files            36       52      +16     
  Lines          4871     5268     +397     
============================================
+ Hits           3677     3922     +245     
- Misses         1194     1346     +152
Impacted Files Coverage Δ Complexity Δ
src/Phinx/Db/Table/ForeignKey.php 97.91% <ø> (ø) 20 <0> (+20) ⬆️
src/Phinx/Db/Table/Index.php 100% <ø> (ø) 13 <0> (+13) ⬆️
src/Phinx/Db/Adapter/SqlServerAdapter.php 0% <0%> (ø) 173 <43> (+173) ⬆️
src/Phinx/Db/Action/AddIndex.php 100% <100%> (ø) 5 <5> (?)
src/Phinx/Db/Action/ChangeColumn.php 100% <100%> (ø) 6 <6> (?)
src/Phinx/Db/Action/RemoveColumn.php 100% <100%> (ø) 3 <3> (?)
src/Phinx/Db/Adapter/ProxyAdapter.php 95% <100%> (+26.25%) 12 <2> (+12) ⬆️
src/Phinx/Db/Plan/AlterTable.php 100% <100%> (ø) 4 <4> (?)
src/Phinx/Db/Action/AddForeignKey.php 100% <100%> (ø) 6 <6> (?)
src/Phinx/Db/Action/Action.php 100% <100%> (ø) 2 <2> (?)
... and 40 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 152a5f1...831d940. Read the comment docs.

@ADmad ADmad removed their request for review April 28, 2018 11:50
*/
namespace Phinx\Db\Action;

abstract class Action
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a convention of having an 'Abstract' prefix to abstract classes? Also,by not having a constructor, this is placing the responsibility of populating the $table on the subclass with no type hinting being enforced in this class. Making $table private and having a protected function setTable(Table $table){$this->table=$table;} would provide that enforcement of the correct type. Or having a constructor and having various tools (PHPStan for example) to identify when/if the subclass doesn't call the parent class

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ll add a constructor, I don’t want setters in the action classes

class AddColumn extends Action
{

protected $column;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typehint

$this->column = $column;
}

public static function build(Table $table, $columnName, $type = null, $options = [])
Copy link
Collaborator

@rquadling rquadling Apr 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should a value object be responsible for being a factory?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don’t see why not. I see it similar to overloading the constructor in java, only that php has no method overloading.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd have an ActionInterface that requires each Action to be able to build itself then.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or an ActionFactory that builds one based upon the supplied data.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any reason in mind for that?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For an object whose main intention is to be passed around once created, itself acting as its own factory is redundant. Once instantiated the objects only purpose is to act as a value object. Not the one hit wonder factory.

The single responsibility aspect is what comes to mind here. Either the object represents some state OR it creates something stateful.

I may be being picky, but if the refactoring is taken in, then a very clear separation would be nicer. Smaller objects, doing fewer things. Far easier to test and understand. But as in all things, compromise between a purists perspective and a practical one.

@broberts-dev
Copy link

This proposed, new internal architecture looks promising, not that I have an opinion that should carry any weight or, at least, not one that was solicited.

I think it would be useful to implement exception handling such that custom exceptions are thrown by each “internals” component/layer (and caught by each preceding layer) flowing all the way back up to the “top level” of the migration execution “call stack”, and leading to concise and intuitive messages about the reason(s) for migration execution failure as part of migration execution as opposed to a fatal runtime error and a “mostly internals” call stack trace. Ideally these messages at the top level would be given in terms of the original/source migration, which seems particularly important with potentially multiple, intermediate forms, translations, optimizations, etc.

Also, the concept of “Actions” within an “Intent” seems to open the door for an automatic, orderly unwinding of partially successful execution of a migration, particularly during development and testing. It’s always quite annoying when, for whatever reason, a failed migration results in a partially-migrated database state.

*
* @param string $tableName
* @param string[] $columns Column(s)
* @param string $constraint Constraint name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

|null here

@ravage84
Copy link
Member

ravage84 commented May 2, 2018

This is a very significant breaking change, and I guess we can have that luxury as phinx has not been marked as stable. From the user point of view, there should be no visible changes, but for anyone extending the internals, this will break their code for sure.

Agree,. Let's do anything that seems necessary to bring Phinx to a stable & robust base for the intermediate future.
In the long run we can always introduce more breaking changes that demand a major bump once we have enough experience, though I'd recommend us to have a plan for easing the upgrade/migration (remember CakePHP 2.x -> 3.x).

*/
protected $table;

public function __construct(Table $table)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing doc comment for function __construct()

* @param mixed $columnName The column name
* @param mixed $type The column type
* @param mixed $options The column options
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing @return tag in function comment

$this->foreignKey = $fk;
}

public static function build(Table $table, $columns, $referencedTable, $referencedColumns = ['id'], array $options = [], $name = null)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing doc comment for function build()

* @param Table $table The table to add the index to
* @param mixed $columns The columns to index
* @param array $options Additional options for the index creation
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing @return tag in function comment

* @param mixed $columnName The name of the column to change
* @param mixed $type The type of the column
* @param mixed $options Addiotional options for the column
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing @return tag in function comment

use Phinx\Db\Action\ChangeColumn;
use Phinx\Db\Action\DropForeignKey;
use Phinx\Db\Action\DropIndex;
use Phinx\Db\Action\DropTable;

This comment was marked as resolved.

use Phinx\Db\Action\DropForeignKey;
use Phinx\Db\Action\DropIndex;
use Phinx\Db\Action\DropTable;
use Phinx\Db\Action\RemoveColumn;

This comment was marked as resolved.

use Phinx\Db\Action\DropIndex;
use Phinx\Db\Action\DropTable;
use Phinx\Db\Action\RemoveColumn;
use Phinx\Db\Action\RenameColumn;

This comment was marked as resolved.

use Phinx\Db\Action\DropTable;
use Phinx\Db\Action\RemoveColumn;
use Phinx\Db\Action\RenameColumn;
use Phinx\Db\Action\RenameTable;

This comment was marked as resolved.

use Phinx\Db\Table\Column;
use Phinx\Db\Table\ForeignKey;
use PHPUnit\Framework\TestCase;
use Phinx\Db\Table\Table;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use classes must be in alphabetical order. Was expecting PHPUnit\Framework\TestCase

@scrutinizer-notifier
Copy link

The inspection completed: 60 new issues, 197 updated code elements

@lorenzo lorenzo merged commit 57e8560 into master May 6, 2018
@lorenzo lorenzo deleted the refactor-internals branch May 6, 2018 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants