dcraft.interface.data package

Submodules

dcraft.interface.data.base module

class dcraft.interface.data.base.DataRepository

Bases: ABC

abstract load(project_name: str, layer_name: str, id: str, format: str, content_type: ContentType) → DataFrame | dict | List[Dict]

Load the specified project, layer, and content based on the given parameters.

Parameters:

  • project_name (str) – The name of the project to load.

  • layer_name (str) – The name of the layer to load.

  • id (str) – The ID of the content to load.

  • format (str) – The format of the content to load.

  • content_type (ContentType) – The type of the content to load.

Returns:

The loaded content of the specified project, layer, and ID.

Return type:

CoveredContentType

abstract save(content: DataFrame | dict | List[Dict], project_name: str, layer_name: str, id: str, format: str, content_type: ContentType)

Save the given content to a specified location.

Parameters:

  • content (CoveredContentType) – The content to be saved.

  • project_name (str) – The name of the project.

  • layer_name (str) – The name of the layer.

  • id (str) – The ID of the content.

  • format (str) – The format of the content.

  • content_type (ContentType) – The type of the content.

Returns:

None

dcraft.interface.data.gcs module

class dcraft.interface.data.gcs.GcsDataRepository(project_id: str, bucket_name: str, credentials: Any | None = None, _http: Any | None = None, client_info: Any | None = None, client_options: Any | None = None, use_auth_w_custom_endpoint: bool = True)

Bases: DataRepository

load(project_name: str, layer_name: str, id: str, format: str, content_type: ContentType) → DataFrame | dict | List[Dict]

Load the content from the specified project, layer, and ID, with the given format and content type.

Parameters:

  • project_name (str) – The name of the project.

  • layer_name (str) – The name of the layer.

  • id (str) – The ID of the content.

  • format (str) – The format of the content.

  • content_type (ContentType) – The type of the content.

Returns:

The loaded content.

Return type:

CoveredContentType

Raises:
save(content: DataFrame | dict | List[Dict], project_name: str, layer_name: str, id: str, format: str, content_type: ContentType)

Save the provided content to the specified project, layer, and ID in the given format and content type.

Parameters:

  • content (CoveredContentType) – The content to be saved.

  • project_name (str) – The name of the project.

  • layer_name (str) – The name of the layer.

  • id (str) – The ID of the content.

  • format (str) – The format in which the content should be saved.

  • content_type (ContentType) – The type of the content.

Raises:
Returns:

None

dcraft.interface.data.local module

class dcraft.interface.data.local.LocalDataRepository(dir_path: str)

Bases: DataRepository

load(project_name: str, layer_name: str, id: str, format: str, content_type: ContentType) → DataFrame | dict | List[Dict]

Load the content from a specified path based on the project name, layer name, id, format, and content type.

Parameters:

  • project_name (str) – The name of the project.

  • layer_name (str) – The name of the layer.

  • id (str) – The ID of the content.

  • format (str) – The format of the content.

  • content_type (ContentType) – The type of the content.

Returns:

The loaded content.

Return type:

CoveredContentType

Raises:
save(content: DataFrame | dict | List[Dict], project_name: str, layer_name: str, id: str, format: str, content_type: ContentType)

Saves the given content to a file with the specified project name, layer name, ID, format, and content type.

Parameters:

  • content (CoveredContentType) – The content to be saved.

  • project_name (str) – The name of the project.

  • layer_name (str) – The name of the layer.

  • id (str) – The ID of the content.

  • format (str) – The format of the file to be saved.

  • content_type (ContentType) – The type of the content.

Raises:
Returns:

None

dcraft.interface.data.minio module

class dcraft.interface.data.minio.MinioRepository(endpoint: str, bucket: str, access_key: str | None = None, secret_key: str | None = None, session_token: str | None = None, secure: bool = True, region: str | None = None, http_client: PoolManager | None = None, credentials: Provider | None = None, cert_check: bool = True)

Bases: DataRepository

load(project_name: str, layer_name: str, id: str, format: str, content_type: ContentType) → DataFrame | dict | List[Dict]

Load the specified content from the given project, layer, and ID.

Parameters:

  • project_name (str) – The name of the project.

  • layer_name (str) – The name of the layer.

  • id (str) – The ID of the content.

  • format (str) – The format of the content.

  • content_type (ContentType) – The type of the content.

Returns:

The loaded content.

Return type:

CoveredContentType

Raises:

ContentExtensionMismatch – If the content cannot be saved with the given extension.

save(content: DataFrame | dict | List[Dict], project_name: str, layer_name: str, id: str, format: str, content_type: ContentType)

Save the content to a specified location in the bucket.

Parameters:

  • content (CoveredContentType) – The content to be saved.

  • project_name (str) – The name of the project.

  • layer_name (str) – The name of the layer.

  • id (str) – The unique identifier.

  • format (str) – The format of the content.

  • content_type (ContentType) – The type of the content.

Raises:

Module contents