Skip to content

COMPONENT_INTEGRATION.md

Latest commit

 

History

History
426 lines (309 loc) · 19.3 KB

COMPONENT_INTEGRATION.md

File metadata and controls

426 lines (309 loc) · 19.3 KB
Raw

Component Integration

Since the ODH operator is the integration point to deploy ODH component manifests, it is essential to have common processes to integrate new components.

The platform supports two integration models:

Model When to use Where the reconciler lives Registration
In-tree component Legacy / not yet modularized internal/controller/components/<name>/ existingComponents in cmd/main.go
Out-of-tree module Preferred for new work and migrations Standalone module-operator repo + thin handler in internal/controller/modules/<name>/ existingModules in cmd/main.go

This document covers in-tree component integration. For out-of-tree modules (for example AI Gateway and Spark Operator), follow internal/controller/modules/README.md instead — embed BaseHandler, implement IsEnabled / BuildModuleCR, and register the handler in existingModules. Do not add a static .Owns() for module CRs on the DSC controller; the modules controller uses dynamic ownership.

To understand the current operator architecture and its inner workings, please refer to the design document.

The list of currently integrated components and modules is provided at the end of this document.

Use scaffolding to create boilerplate code

Integrating a new in-tree component into the Open Data Hub (ODH) operator is easier with the component-codegen CLI. The CLI automates much of the boilerplate code and file generation, significantly reducing manual effort and ensuring consistency.

While the CLI handles most of the heavy lifting, it’s still important to understand the purpose of each generated file. Please refer to the following sections for a detailed breakdown of the key files and their roles in the integration process.

Integrating a new in-tree component

To ensure a new in-tree component is integrated seamlessly in the operator, please follow the steps listed below.

1. Update API specs

The first step is to define the internal API spec for the new component and introduce it to the existing DataScienceCluster (DSC) API. Please proceed as follows:

Define internal API spec for the new component

  1. Create a dedicated <example_component_name>_types.go file within api/component/v1alpha1 directory.

  2. Define the internal API spec for the new component according to the expected definitions. You can use the following pseudo-implementation for reference:

package v1alpha1

import (
	"github.com/opendatahub-io/opendatahub-operator/v2/api/common"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

const (
  	// example new component name
	ExampleComponentName         = "examplecomponent"

	// ExampleComponentInstanceName is the name of the new component instance singleton
	// value should match what is set in the kubebuilder markers for XValidation defined below
	ExampleComponentInstanceName = "default-examplecomponent"

  	// kubernetes kind of the new component
	ExampleComponentKind         = "ExampleComponent"
)

type ExampleComponentCommonSpec struct {
	// new component spec shared with DSC api
  	// ( refer/define here if applicable to the new component )
}

// ExampleComponentSpec defines the desired state of ExampleComponent
type ExampleComponentSpec struct {
	// new component spec exposed to DSC api
	ExampleComponentCommonSpec `json:",inline"`

	// new component spec exposed only to internal api
	// IMPORTANT: fields here must only be written by the operator itself (e.g. values
	// discovered at run time based on cluster config). Never place user-facing configuration
	// here — anything a user must set belongs in ExampleComponentCommonSpec so it is
	// reachable through the DSC API. Internal CRDs are hidden from the OperatorHub UI
	// and are not the expected user-facing API surface.
  	// ( refer/define here if applicable to the new component )
}

// ExampleComponentCommonStatus defines the shared observed state of ExampleComponent
type ExampleComponentCommonStatus struct {
	// add fields/attributes if needed
}

// ExampleComponentStatus defines the observed state of ExampleComponent
type ExampleComponentStatus struct {
	common.Status `json:",inline"`
	ExampleComponentCommonStatus `json:",inline"`
}

// default kubebuilder markers for the new component
// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:resource:scope=Cluster
// +kubebuilder:validation:XValidation:rule="self.metadata.name == 'default-examplecomponent'",message="ExampleComponent name must be default-examplecomponent"
// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status`,description="Ready"
// +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason`,description="Reason"

// ExampleComponent is the Schema for the new component API
type ExampleComponent struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   ExampleComponentSpec   `json:"spec,omitempty"`
	Status ExampleComponentStatus `json:"status,omitempty"`
}

// status getter
func (c *ExampleComponent) GetStatus() *common.Status {
	return &c.Status.Status
}

func (c *ExampleComponent) GetConditions() []common.Condition {
	return c.Status.GetConditions()
}

func (c *ExampleComponent) SetConditions(conditions []common.Condition) {
	c.Status.SetConditions(conditions)
}

func (c *ExampleComponent) GetReleaseStatus() *[]common.ComponentRelease {
	return &c.Status.Releases
}

func (c *ExampleComponent) SetReleaseStatus(releases []common.ComponentRelease) {
	c.Status.Releases = releases
}

// +kubebuilder:object:root=true

// ExampleComponentList contains a list of ExampleComponent
type ExampleComponentList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []ExampleComponent `json:"items"`
}

// register the defined schemas
func init() {
	SchemeBuilder.Register(&ExampleComponent{}, &ExampleComponentList{})
}

// DSCExampleComponent contains all the configuration exposed in DSC instance for ExampleComponent component
// ( utilize DSC prefix here for naming consistency with the other integrated components )
type DSCExampleComponent struct {
	// configuration fields common across components
	common.ManagementSpec `json:",inline"`

	// new component-specific fields
	ExampleComponentCommonSpec `json:",inline"`
}

// DSCExampleComponentStatus struct holds the status for the ExampleComponent component exposed in the DSC
type DSCExampleComponentStatus struct {
	common.ManagementSpec    `json:",inline"`
	*ExampleComponentCommonStatus `json:",inline"`
}

Alternatively, you can refer to the existing integrated component APIs located within api/component/v1alpha1 directory.

Define internal resources for the new component

Component CRDs can be marked as internal to hide them from users in the OperatorHub UI. Internal component CRDs are not the expected user-facing API surface — the DataScienceCluster (DSC) is. Any configuration field that a user must set must be surfaced through the DSC spec via DSCExampleComponent (using ExampleComponentCommonSpec). Fields that live only on the internal ExampleComponentSpec (outside of CommonSpec) must be values written exclusively by the operator itself, such as infrastructure details propagated from GatewayConfig. Most component resources should be internal; this means users are NOT expected to edit them directly.

To mark your component as internal, update config/manifests/bases/opendatahub-operator.clusterserviceversion.yaml and config/rhoai/manifests/bases/rhods-operator.clusterserviceversion.yamlto add your component's CRD to the operators.operatorframework.io/internal-objects annotation:

operators.operatorframework.io/internal-objects: '["dashboards.components.platform.opendatahub.io",
  "datasciencepipelines.components.platform.opendatahub.io",
  ...
  "examplecomponents.components.platform.opendatahub.io"]'

Add component to the owned objects list

Add your component to the resources list in PROJECT file (if you used component-codegen CLI, it should have been already added):

    resources:
    - api:
        crdVersion: v1alpha1
      controller: true
      domain: platform.opendatahub.io
      group: components
      kind: ExampleComponent
      path: github.com/opendatahub-io/opendatahub-operator/v2/api/components/v1alpha1
      version: v1alpha1

and run make bundle-all to verify your component has been correctly added to config/manifests/bases/opendatahub-operator.clusterserviceversion.yaml and config/rhoai/manifests/bases/rhods-operator.clusterserviceversion.yaml.

Add Component to DataScienceCluster API spec

DataScienceCluster (DSC) CRD is responsible for enabling individual components and exposing them to end users. To introduce the newly defined component API, extend the Components struct within the DataScienceCluster API spec (located within api/datasciencecluster/v1) to include the new API.

type Components struct {
	// Dashboard component configuration.
	Dashboard componentApi.DSCDashboard `json:"dashboard,omitempty"`

	// Workbenches component configuration.
	Workbenches componentApi.DSCWorkbenches `json:"workbenches,omitempty"`

	// ... other currently integrated components ...

	// add the new component as follows
+	ExampleComponent componentApi.DSCExampleComponent `json:"examplecomponent,omitempty"`
}

Additionally, extend the ComponentsStatus struct within the same file to include the new component status to be exposed in the DSC.

// ComponentsStatus defines the custom status of DataScienceCluster components.
type ComponentsStatus struct {
	// Dashboard component status.
	Dashboard componentApi.DSCDashboardStatus `json:"dashboard,omitempty"`

	// Workbenches component status.
	Workbenches componentApi.DSCWorkbenchesStatus `json:"workbenches,omitempty"`

	// ... other currently integrated component statuses ...

	// add the new component status as follows
+	ExampleComponent componentApi.DSCExampleComponentStatus `json:"examplecomponent,omitempty"`
}

Update kubebuilder_rbac.go

Add kubebuilder RBAC permissions intended for the new component into internal/controller/datasciencecluster/kubebuilder_rbac.go.

Update the dependent files

To fully reflect the API changes brought by the addition of the new component, run the following command:

make generate manifests api-docs bundle

This command will (re-)generate the necessary kubebuilder functions, and update both the API documentation and the operator bundle manifests.

2. Create an in-tree package for the new component reconciliation logic

To add new component-specific reconciler logic, create a dedicated <example_component_name> package under internal/controller/components. For reference, that directory contains reconciler implementations for the currently integrated in-tree components.

Note: This section is for in-tree components only. For out-of-tree modules, do not add a controller under internal/controller/components. Implement a ModuleHandler under internal/controller/modules/ instead (see modules README).

Implement the component handler interface

Each in-tree component that is intended to be managed by the operator is expected to be included in the components registry. The components registry defines a component handler interface which is required to be implemented for the new component. To do so, create a dedicated <example_component_name>.go file within the newly created component package and provide the interface implementation:

type componentHandler struct{}

func init() { //nolint:gochecknoinits
	cr.Add(&componentHandler{})
}

func (s *componentHandler) GetName() string

func (s *componentHandler) GetManagementState(dsc *dscv1.DataScienceCluster) operatorv1.ManagementState

func (s *componentHandler) NewCRObject(dsc *dscv1.DataScienceCluster) common.PlatformObject

func (s *componentHandler) Init(platform common.Platform) error

func (s *componentHandler) UpdateDSCStatus(ctx context.Context, rr *types.ReconciliationRequest) (metav1.ConditionStatus, error)

Please refer the existing component implementations in the internal/controller/components directory for further details.

Implement new component reconciler

Create a dedicated <example_component_name>_controller.go file and implement the expected NewComponentReconciler function there. This function will be responsible for creating the reconciler for the previously introduced <ExampleComponent> API.

NewControllerReconciler utilizes a generic builder pattern, that supports defining various types of relationships and functionality:

  • resource ownership - using .Owns()
  • watching a resource - using .Watches()
  • reconciler actions - using .WithAction()
    • this includes pre-implemented actions used commonly across components (e.g. manifests rendering), as well as customized, component-specific actions
    • more details on actions are provided below

The example pseudo-implementation should look like as follows:

func (s *componentHandler) NewComponentReconciler(ctx context.Context, mgr ctrl.Manager) error {
	_, err := reconciler.ReconcilerFor(mgr, &componentApi.ExampleComponent{}).
		Owns(...).
		// ... add other necessary resource ownerships
		Watches(...).
		// ... add other necessary resource watches
		WithAction(...).
		// ... add custom actions if needed
		// ... add mandatory common actions (e.g. manifest rendering, deployment, garbage collection)
		Build(ctx)

	if err != nil {
		return err
	}

	return nil
}
Actions

Actions are functions that define pieces of component reconciliation logic. Any action is expected to conform to the following signature:

func exampleAction(ctx context.Context, rr *odhtypes.ReconciliationRequest) error

Such actions can be then introduced to the reconciler builder using .WithAction() calls. As seen in the existing component reconciler implementations, it would be recommended to include the action implementations in a separate file within the package, such as <example_component_name>_controller_actions.go.

"Generic"/commonly-implemented actions for each of the currently integrated components include:

  • initialize() - to register paths to the component manifests

In addition, proper generic actions, intended to be used across the components, are provided as part of the operator implementation (located in pkg/controller/actions). These support:

  • manifest rendering
    • can additionally utilize caching
  • manifest deployment
    • can additionally utilize caching
  • status updating
  • garbage collection
    • additional requirement - garbage collection action must always be called as the last action before the final .Build() call

If the new component requires additional custom logic, custom actions can also be added to the builder via the respective .WithAction() calls.

For practical examples of all the above-mentioned functionality, please refer to the implementations within internal/controller/components directory.

Update main.go

Import the newly added component:

package main

import (
	// ... existing imports ...

	// ... component imports for the integrated components ...
+	_ "github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/<example_component>"
)

3. Add unit and e2e tests

Please add unit tests for any component-specific functions added to the codebase.

Please also add e2e tests to the e2e test suite to capture deployments introduced by the new component. Existing e2e test suites for the integrated components can be also found there.

Lastly, please update the following files to fully integrate new component tests into the overall test suite:

  • update CreateDSC() function in tests/e2e/helper_test.go to set new component in DSC
  • update componentsTestSuites map in tests/e2e/controller_test.go to include the reference for the new component e2e test suite
  • add a components.<name> entry to tests/e2e/scripts/e2e-scope-rules.yaml, so path-based e2e test scoping (see docs/e2e-testing.md) knows about the new component. cmd/main_test.go and tests/e2e/e2e_scope_rules_registry_test.go fail if this is missing, though see that doc's follow-up gap note on when those tests actually run.

4. Update Prometheus config and tests

If the component is planned to be released for downstream, Prometheus rules and promtest need to be updated for the component.

  • Rules are located in ./internal/controller/components/<component>/monitoring/<component>-prometheusrules.tmpl.yaml file
  • Tests are grouped in tests/prometheus_unit_tests _unit_tests.yam file

5. Update CRD Kustomization reference

Add new component CRD reference in ./config/crd/kustomization.yaml file.

6. Update list of owned/watched types by DataScienceCluster reconciler

For in-tree components, adjust NewDataScienceClusterReconciler in ./internal/controller/datasciencecluster/datasciencecluster_controller.go to .Owns() the component CR.

For out-of-tree modules, skip this step. Module CRs are owned via the modules controller's dynamic ownership path (see modules README).

Integrated components

In-tree components

Controller implementations live under internal/controller/components/. APIs live under api/components/v1alpha1/.

Out-of-tree modules

Platform handlers live under internal/controller/modules/. The workload / module operators live in their own repositories. See modules README.