Issue#51/support operations

[jonasz-lasut]

Fixes #51
Added support for Operations to existing queries. It may be used together with CronOperation or WatchOperation to force updates of queries in critical instances of XRs based on name/labels. For example composition by default does not forcefully update the query result to avoid throttling but for XRs with label always-update you can create WatchOperation which calls function-msgraph with skipQueryWhenTargetHasData: false.

Added Operations' examples, unit tests and tested manually on UP UXP 2.0.2-up.3. Function package was built with crossplane-cli v2.0.2.

The last update was triggered via Operation and forcefully updated

I have:

• Read and followed Crossplane's contribution process.
• Added or updated unit tests for my change.

[ytsarev]

@jonasz-lasut if I understand correctly, the result of Operation will be visible only in the logs, is that correct?

If yes, can we extend the operation code path to write the informative data to XR metadata? I found it very useful to expose the function operation data in https://github.com/upbound/configuration-aws-database-ai/blob/main/operations/rds-intelligent-scaling-cron/operation.yaml#L63-L67 ( function-claude ai example)

In this case we can make it deterministically in the code and not exclusively rely on the log stream

[jonasz-lasut]

@ytsarev I've implemented the discussed changes and added self-healing mechanism based on the query-drift annotation

I've created an XR with group validation, added CronOperation with 1 min interval. No drift was detected at first.
Then I've added group description in Entra, another operation run has set the drift annotation to true and next time XR was reconciled it didn't skip query even though function input in Composition is set to true (self-heal). Another minute has passed and CronOperation reconciled annotation to drift false.

Yamls:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xrs.example.crossplane.io
spec:
  group: example.crossplane.io
  names:
    categories:
    - crossplane
    kind: XR
    plural: xrs
  versions:
  - name: v1
    referenceable: true
    schema:
      openAPIV3Schema:
        description: XR is the Schema for the XR API.
        properties:
          spec:
            description: XRSpec defines the desired state of XR.
            type: object
            properties:
              queryResourceType:
                description: resource type for az resource query construction
                type: string
              groupConfig:
                description: Configuration for group references
                type: object
                properties:
                  name:
                    description: Name of a single group to reference with groupRef
                    type: string
                  names:
                    description: List of group names to reference with groupsRef
                    type: array
                    items:
                      type: string
              userAccess:
                description: Configuration for user references
                type: object
                properties:
                  emails:
                    description: List of user emails to reference with usersRef
                    type: array
                    items:
                      type: string
              servicePrincipalConfig:
                description: Configuration for service principal references
                type: object
                properties:
                  names:
                    description: List of service principal names to reference with servicePrincipalsRef
                    type: array
                    items:
                      type: string
          status:
            description: XRStatus defines the observed state of XR.
            type: object
            properties:
              groupMembers:
                description: Freeform field containing query results from function-msgraph
                type: array
                items:
                  type: object
                x-kubernetes-preserve-unknown-fields: true
              validatedUsers:
                description: Freeform field containing query results from function-msgraph
                type: array
                items:
                  type: object
                x-kubernetes-preserve-unknown-fields: true
              groupObjectIDs:
                description: Freeform field containing query results from function-msgraph
                type: array
                items:
                  type: object
                x-kubernetes-preserve-unknown-fields: true
              servicePrincipals:
                description: Freeform field containing query results from function-msgraph
                type: array
                items:
                  type: object
                x-kubernetes-preserve-unknown-fields: true
        required:
        - spec
        type: object
    served: true
status:
  controllers:
    compositeResourceClaimType:
      apiVersion: ""
      kind: ""
    compositeResourceType:
      apiVersion: ""
      kind: ""
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: user-validation-example-status-ref
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1
    kind: XR
  mode: Pipeline
  pipeline:
  - step: get-group-objectids
    functionRef:
      name: function-msgraph
    input:
      apiVersion: msgraph.fn.crossplane.io/v1alpha1
      kind: Input
      queryType: GroupObjectIDs
      groups:
        - "TestGroup"
      target: "status.groupObjectIDs"
      skipQueryWhenTargetHasData: true
    credentials:
      - name: azure-creds
        source: Secret
        secretRef:
          namespace: upbound-system
          name: provider-secret
---
# Example XR with both spec references and status fields
apiVersion: example.crossplane.io/v1
kind: XR
metadata:
  name: example-xr
spec: {}
---
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-msgraph
spec:
  package: ttl.sh/function-msgraph:v0.4.0-operations-rc12
---
apiVersion: ops.crossplane.io/v1alpha1
kind: CronOperation
metadata:
  name: update-user-validation-for-critical-xr
spec:
  schedule: "*/1 * * * *" # Every minute
  concurrencyPolicy: Forbid
  successfulHistoryLimit: 1
  failedHistoryLimit: 1
  operationTemplate:
    spec:
      mode: Pipeline
      pipeline:
      - step: user-validation
        functionRef:
          name: function-msgraph
        input:
          apiVersion: msgraph.fn.crossplane.io/v1alpha1
          kind: Input
          queryType: GroupObjectIDs
          groups:
            - "TestGroup"
          target: "status.groupObjectIDs"
        credentials:
          - name: azure-creds
            source: Secret
            secretRef:
              namespace: upbound-system
              name: provider-secret
        requirements:
          requiredResources:
          - requirementName: ops.crossplane.io/watched-resource
            apiVersion: example.crossplane.io/v1
            kind: XR
            name: example-xr

[ytsarev]

I tested the solution e2e with xpkg.upbound.io/upbound/function-msgraph:v0.4.0-rc1 build and it works

k get xrs.example.crossplane.io example-xr -o yaml
apiVersion: example.crossplane.io/v1
kind: XR
metadata:
  annotations:
    function-msgraph/last-execution: "2025-09-09T16:26:01Z"
    function-msgraph/last-execution-query-drift-detected: "false"

Drift is properly getting detected by Operations, annotation getting set to true and the Composition gets XR status updated to the new query result.

  - description: test group for https://github.com/upbound/function-msgraph hello
      test2
    displayName: test-fn-msgraph
    id: 4bd24f32-d98b-4563-9c2e-7083dd7ae4

Possible and optional suggestion is to extend verbosity of Operation event stream

k describe operations.ops.crossplane.io update-user-validation-for-critical-xr-1757435280
...
Events:
  Type    Reason           Age   From                             Message
  ----    ------           ----  ----                             -------
  Normal  RunPipelineStep  14s   ops/operation.ops.crossplane.io  Pipeline step "user-validation": QueryType: "GroupObjectIDs"

Currently it just logs QueryType there, we probably can be more verbose there, but I leave it totally up to you and maybe to additional PR.

Please see inline suggestion for README extension, otherwise PR is approved.

[jonasz-lasut]

I've applied the suggested README changes.
Re logging: I was not sure if verbose logging would not be an issue when combined with periodical nature of Operations, I'll be happy to extend the verbosity when implementing the admin consent grant as it'll be first Operation-first capability of this function