Skip to content

core,graph,store: on block revert, Firehose should save the cursor to the database#3133

Merged
maoueh merged 1 commit intomasterfrom
maoueh/fix/firehose-save-cursor-on-revert
Jan 20, 2022
Merged

core,graph,store: on block revert, Firehose should save the cursor to the database#3133
maoueh merged 1 commit intomasterfrom
maoueh/fix/firehose-save-cursor-on-revert

Conversation

@maoueh
Copy link
Copy Markdown
Contributor

@maoueh maoueh commented Jan 7, 2022

This PR ensures that the Firehose cursor is saved properly even on revert operations.

Copy link
Copy Markdown
Collaborator

@leoyvens leoyvens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For all I can tell, we can refactor BlockStreamEvent::Revert to always require a parent block ptr, rather than taking an Option.

@maoueh
Copy link
Copy Markdown
Contributor Author

maoueh commented Jan 7, 2022

Better than that, it was actually already fetched once in PollingBlockStream and passed, so my change would have break that.

The code in instance manager is not required anymore actually.

@maoueh
Copy link
Copy Markdown
Contributor Author

maoueh commented Jan 7, 2022

Done.

Copy link
Copy Markdown
Collaborator

@leoyvens leoyvens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All DB operations to process a revert should be done in the same DB transaction, so we can rely on ACID guarantees. So the update_block_cursor operation should be done in the same transaction as the other operations.

@maoueh
Copy link
Copy Markdown
Contributor Author

maoueh commented Jan 10, 2022

@leoyvens You are right, makes total sense. I've updated the PR.

While doing the operations, while updating where revert_block_operations is called, I noticed to other places where right now Firehose is not handled:

  • graphman rewind command
  • unfail_deterministic_error does some revert of block operations and should handle the firehose cursor also. I would like to discuss this in a chat with you today.

@leoyvens
Copy link
Copy Markdown
Collaborator

For the purpose of this PR, we need to at least understand if it's safe to call fn revert_block_operations with a None cursor, as is being done in those two places.

… the databse

- On revert operation, `parent_ptr` everywhere since it avoids a useless RPC (or in cache database) call.
- The firehose update cursor on remove is directly in `revert_block_operations` to be in a transaction alongside the update of the subgraph ptr.
@maoueh maoueh force-pushed the maoueh/fix/firehose-save-cursor-on-revert branch from 845432d to 43a3166 Compare January 19, 2022 19:18
@maoueh
Copy link
Copy Markdown
Contributor Author

maoueh commented Jan 19, 2022

As discussed last week, for rewind and unfail_determinisc_error we are good with just actually removing the cursor completely.

If an upcoming PR, I'm going to make it possible for firehose to start from a block_ptr and ensure it's the correct canonical block we seek.

I think the PR is good to merge in it's current shape and next improvement will come in a subsequent PR.

conn: &PgConnection,
site: Arc<Site>,
block_ptr_to: BlockPtr,
firehose_cursor: Option<&str>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this still be an Option, given this is now using the empty string?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I used "" at some place to "clear" the cursor, because right now None means do nothing (which is the case when using Ethereum RPC).

So I'm of the opinion that we should keep it like this.

@maoueh maoueh merged commit f4c4980 into master Jan 20, 2022
@maoueh maoueh deleted the maoueh/fix/firehose-save-cursor-on-revert branch January 20, 2022 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants