Metadata Templates

API: Internal/Beta

Metadata templates define the schema for metadata documents.

Rationale

📝 NOTE: This API follows the standard Resources API. We recommend that you have already read and understood the concepts described here.


UCloud supports arbitrary of files. This feature is useful for general data management. It allows users to tag documents at a glance and search through them.

This feature consists of two parts:

  1. Metadata templates (you are here): Templates specify the schema. You can think of this as a way of defining how your documents should look. We use them to generate user interfaces and visual representations of your documents.

  2. Metadata documents (next section): Documents fill out the values of a template. When you create a document you must attach it to a file also.

At a technical level, we implement metadata templates using JSON schema. This gives you a fair amount of flexibility to control the format of documents. Of course, not everything is machine-checkable. To mitigate this, templates can require that changes go through an approval process. Only administrators of a workspace can approve such changes.

Table of Contents

1. Examples
Description
The Sensitivity Template
2. Remote Procedure Calls
Name Description
browse Browses the catalog of available resources
browseTemplates No description
retrieve Retrieve a single resource
retrieveLatest No description
retrieveProducts Retrieve product support for all accessible providers
retrieveTemplate No description
create Creates one or more resources
createTemplate No description
deprecate No description
init Request (potential) initialization of resources
updateAcl Updates the ACL attached to a resource
3. Data Models
Name Description
FileMetadataTemplateAndVersion No description
FileMetadataTemplateNamespace A `Resource` is the core data model used to synchronize tasks between UCloud and Provider.
FileMetadataTemplateNamespace.Spec No description
FileMetadataTemplateNamespace.Status Describes the current state of the `Resource`
FileMetadataTemplateNamespace.Update Describes an update to the `Resource`
FileMetadataTemplateNamespaceFlags No description
FileMetadataTemplateSupport No description
FileMetadataTemplatesBrowseTemplatesRequest The base type for requesting paginated content.

Example: The Sensitivity Template

Frequency of useCommon
Actors
  • An authenticated user (user)
Communication Flow: Kotlin
FileMetadataTemplateNamespaces.createTemplate.call(
    bulkRequestOf(FileMetadataTemplate(
        changeLog = "Initial version", 
        createdAt = 0, 
        description = "File sensitivity for files", 
        inheritable = true, 
        namespaceId = "sensitivity", 
        namespaceName = null, 
        namespaceType = FileMetadataTemplateNamespaceType.COLLABORATORS, 
        requireApproval = true, 
        schema = JsonObject(mapOf("type" to JsonLiteral(
            content = "object", 
            isString = true, 
        )),"title" to JsonLiteral(
            content = "UCloud File Sensitivity", 
            isString = true, 
        )),"required" to listOf(JsonLiteral(
            content = "sensitivity", 
            isString = true, 
        ))),"properties" to JsonObject(mapOf("sensitivity" to JsonObject(mapOf("enum" to listOf(JsonLiteral(
            content = "SENSITIVE", 
            isString = true, 
        ), JsonLiteral(
            content = "CONFIDENTIAL", 
            isString = true, 
        ), JsonLiteral(
            content = "PRIVATE", 
            isString = true, 
        ))),"type" to JsonLiteral(
            content = "string", 
            isString = true, 
        )),"title" to JsonLiteral(
            content = "File Sensitivity", 
            isString = true, 
        )),"enumNames" to listOf(JsonLiteral(
            content = "Sensitive", 
            isString = true, 
        ), JsonLiteral(
            content = "Confidential", 
            isString = true, 
        ), JsonLiteral(
            content = "Private", 
            isString = true, 
        ))),))),))),"dependencies" to JsonObject(mapOf())),)), 
        title = "Sensitivity", 
        uiSchema = JsonObject(mapOf("ui:order" to listOf(JsonLiteral(
            content = "sensitivity", 
            isString = true, 
        ))),)), 
        version = "1.0.0", 
    )),
    user
).orThrow()

/*
BulkResponse(
    responses = listOf(FileMetadataTemplateAndVersion(
        id = "15123", 
        version = "1.0.0", 
    )), 
)
*/
FileMetadataTemplateNamespaces.retrieveLatest.call(
    FindByStringId(
        id = "15123", 
    ),
    user
).orThrow()

/*
FileMetadataTemplate(
    changeLog = "Initial version", 
    createdAt = 0, 
    description = "File sensitivity for files", 
    inheritable = true, 
    namespaceId = "sensitivity", 
    namespaceName = null, 
    namespaceType = FileMetadataTemplateNamespaceType.COLLABORATORS, 
    requireApproval = true, 
    schema = JsonObject(mapOf("type" to JsonLiteral(
        content = "object", 
        isString = true, 
    )),"title" to JsonLiteral(
        content = "UCloud File Sensitivity", 
        isString = true, 
    )),"required" to listOf(JsonLiteral(
        content = "sensitivity", 
        isString = true, 
    ))),"properties" to JsonObject(mapOf("sensitivity" to JsonObject(mapOf("enum" to listOf(JsonLiteral(
        content = "SENSITIVE", 
        isString = true, 
    ), JsonLiteral(
        content = "CONFIDENTIAL", 
        isString = true, 
    ), JsonLiteral(
        content = "PRIVATE", 
        isString = true, 
    ))),"type" to JsonLiteral(
        content = "string", 
        isString = true, 
    )),"title" to JsonLiteral(
        content = "File Sensitivity", 
        isString = true, 
    )),"enumNames" to listOf(JsonLiteral(
        content = "Sensitive", 
        isString = true, 
    ), JsonLiteral(
        content = "Confidential", 
        isString = true, 
    ), JsonLiteral(
        content = "Private", 
        isString = true, 
    ))),))),))),"dependencies" to JsonObject(mapOf())),)), 
    title = "Sensitivity", 
    uiSchema = JsonObject(mapOf("ui:order" to listOf(JsonLiteral(
        content = "sensitivity", 
        isString = true, 
    ))),)), 
    version = "1.0.0", 
)
*/
FileMetadataTemplateNamespaces.browseTemplates.call(
    FileMetadataTemplatesBrowseTemplatesRequest(
        consistency = null, 
        id = "15123", 
        itemsPerPage = null, 
        itemsToSkip = null, 
        next = null, 
    ),
    user
).orThrow()

/*
PageV2(
    items = listOf(FileMetadataTemplate(
        changeLog = "Initial version", 
        createdAt = 0, 
        description = "File sensitivity for files", 
        inheritable = true, 
        namespaceId = "sensitivity", 
        namespaceName = null, 
        namespaceType = FileMetadataTemplateNamespaceType.COLLABORATORS, 
        requireApproval = true, 
        schema = JsonObject(mapOf("type" to JsonLiteral(
            content = "object", 
            isString = true, 
        )),"title" to JsonLiteral(
            content = "UCloud File Sensitivity", 
            isString = true, 
        )),"required" to listOf(JsonLiteral(
            content = "sensitivity", 
            isString = true, 
        ))),"properties" to JsonObject(mapOf("sensitivity" to JsonObject(mapOf("enum" to listOf(JsonLiteral(
            content = "SENSITIVE", 
            isString = true, 
        ), JsonLiteral(
            content = "CONFIDENTIAL", 
            isString = true, 
        ), JsonLiteral(
            content = "PRIVATE", 
            isString = true, 
        ))),"type" to JsonLiteral(
            content = "string", 
            isString = true, 
        )),"title" to JsonLiteral(
            content = "File Sensitivity", 
            isString = true, 
        )),"enumNames" to listOf(JsonLiteral(
            content = "Sensitive", 
            isString = true, 
        ), JsonLiteral(
            content = "Confidential", 
            isString = true, 
        ), JsonLiteral(
            content = "Private", 
            isString = true, 
        ))),))),))),"dependencies" to JsonObject(mapOf())),)), 
        title = "Sensitivity", 
        uiSchema = JsonObject(mapOf("ui:order" to listOf(JsonLiteral(
            content = "sensitivity", 
            isString = true, 
        ))),)), 
        version = "1.0.0", 
    )), 
    itemsPerPage = 50, 
    next = null, 
)
*/
FileMetadataTemplateNamespaces.browse.call(
    ResourceBrowseRequest(
        consistency = null, 
        flags = FileMetadataTemplateNamespaceFlags(
            filterCreatedAfter = null, 
            filterCreatedBefore = null, 
            filterCreatedBy = null, 
            filterIds = null, 
            filterName = null, 
            filterProductCategory = null, 
            filterProductId = null, 
            filterProvider = null, 
            filterProviderIds = null, 
            hideProductCategory = null, 
            hideProductId = null, 
            hideProvider = null, 
            includeOthers = false, 
            includeProduct = false, 
            includeSupport = false, 
            includeUpdates = false, 
        ), 
        itemsPerPage = null, 
        itemsToSkip = null, 
        next = null, 
        sortBy = null, 
        sortDirection = SortDirection.ascending, 
    ),
    user
).orThrow()

/*
PageV2(
    items = listOf(FileMetadataTemplateNamespace(
        createdAt = 1635151675465, 
        id = "15123", 
        owner = ResourceOwner(
            createdBy = "user", 
            project = null, 
        ), 
        permissions = ResourcePermissions(
            myself = listOf(Permission.ADMIN), 
            others = emptyList(), 
        ), 
        specification = FileMetadataTemplateNamespace.Spec(
            name = "sensitivity", 
            namespaceType = FileMetadataTemplateNamespaceType.COLLABORATORS, 
            product = ProductReference(
                category = "", 
                id = "", 
                provider = "ucloud_core", 
            ), 
        ), 
        status = FileMetadataTemplateNamespace.Status(
            latestTitle = "Sensitivity", 
            resolvedProduct = null, 
            resolvedSupport = null, 
        ), 
        updates = emptyList(), 
        providerGeneratedId = "15123", 
    )), 
    itemsPerPage = 50, 
    next = null, 
)
*/
Communication Flow: Curl
# ------------------------------------------------------------------------------------------------------
# $host is the UCloud instance to contact. Example: 'http://localhost:8080' or 'https://cloud.sdu.dk'
# $accessToken is a valid access-token issued by UCloud
# ------------------------------------------------------------------------------------------------------

# Authenticated as user
curl -XPOST -H "Authorization: Bearer $accessToken" -H "Content-Type: content-type: application/json; charset=utf-8" "$host/api/files/metadataTemplates/templates" -d '{
    "items": [
        {
            "namespaceId": "sensitivity",
            "title": "Sensitivity",
            "version": "1.0.0",
            "schema": {
                "type": "object",
                "title": "UCloud File Sensitivity",
                "required": [
                    "sensitivity"
                ],
                "properties": {
                    "sensitivity": {
                        "enum": [
                            "SENSITIVE",
                            "CONFIDENTIAL",
                            "PRIVATE"
                        ],
                        "type": "string",
                        "title": "File Sensitivity",
                        "enumNames": [
                            "Sensitive",
                            "Confidential",
                            "Private"
                        ]
                    }
                },
                "dependencies": {
                }
            },
            "inheritable": true,
            "requireApproval": true,
            "description": "File sensitivity for files",
            "changeLog": "Initial version",
            "namespaceType": "COLLABORATORS",
            "uiSchema": {
                "ui:order": [
                    "sensitivity"
                ]
            },
            "namespaceName": null,
            "createdAt": 0
        }
    ]
}'


# {
#     "responses": [
#         {
#             "id": "15123",
#             "version": "1.0.0"
#         }
#     ]
# }

curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/metadataTemplates/retrieveLatest?id=15123" 

# {
#     "namespaceId": "sensitivity",
#     "title": "Sensitivity",
#     "version": "1.0.0",
#     "schema": {
#         "type": "object",
#         "title": "UCloud File Sensitivity",
#         "required": [
#             "sensitivity"
#         ],
#         "properties": {
#             "sensitivity": {
#                 "enum": [
#                     "SENSITIVE",
#                     "CONFIDENTIAL",
#                     "PRIVATE"
#                 ],
#                 "type": "string",
#                 "title": "File Sensitivity",
#                 "enumNames": [
#                     "Sensitive",
#                     "Confidential",
#                     "Private"
#                 ]
#             }
#         },
#         "dependencies": {
#         }
#     },
#     "inheritable": true,
#     "requireApproval": true,
#     "description": "File sensitivity for files",
#     "changeLog": "Initial version",
#     "namespaceType": "COLLABORATORS",
#     "uiSchema": {
#         "ui:order": [
#             "sensitivity"
#         ]
#     },
#     "namespaceName": null,
#     "createdAt": 0
# }

curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/metadataTemplates/browseTemplates?id=15123" 

# {
#     "itemsPerPage": 50,
#     "items": [
#         {
#             "namespaceId": "sensitivity",
#             "title": "Sensitivity",
#             "version": "1.0.0",
#             "schema": {
#                 "type": "object",
#                 "title": "UCloud File Sensitivity",
#                 "required": [
#                     "sensitivity"
#                 ],
#                 "properties": {
#                     "sensitivity": {
#                         "enum": [
#                             "SENSITIVE",
#                             "CONFIDENTIAL",
#                             "PRIVATE"
#                         ],
#                         "type": "string",
#                         "title": "File Sensitivity",
#                         "enumNames": [
#                             "Sensitive",
#                             "Confidential",
#                             "Private"
#                         ]
#                     }
#                 },
#                 "dependencies": {
#                 }
#             },
#             "inheritable": true,
#             "requireApproval": true,
#             "description": "File sensitivity for files",
#             "changeLog": "Initial version",
#             "namespaceType": "COLLABORATORS",
#             "uiSchema": {
#                 "ui:order": [
#                     "sensitivity"
#                 ]
#             },
#             "namespaceName": null,
#             "createdAt": 0
#         }
#     ],
#     "next": null
# }

curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/metadataTemplates/browse?includeOthers=false&includeUpdates=false&includeSupport=false&includeProduct=false&sortDirection=ascending" 

# {
#     "itemsPerPage": 50,
#     "items": [
#         {
#             "id": "15123",
#             "specification": {
#                 "name": "sensitivity",
#                 "namespaceType": "COLLABORATORS",
#                 "product": {
#                     "id": "",
#                     "category": "",
#                     "provider": "ucloud_core"
#                 }
#             },
#             "createdAt": 1635151675465,
#             "status": {
#                 "latestTitle": "Sensitivity",
#                 "resolvedSupport": null,
#                 "resolvedProduct": null
#             },
#             "updates": [
#             ],
#             "owner": {
#                 "createdBy": "user",
#                 "project": null
#             },
#             "permissions": {
#                 "myself": [
#                     "ADMIN"
#                 ],
#                 "others": [
#                 ]
#             }
#         }
#     ],
#     "next": null
# }
Communication Flow: Visual

Remote Procedure Calls

browse

API: Internal/Beta Auth: Users

Browses the catalog of available resources

Request Response Error
ResourceBrowseRequest<FileMetadataTemplateNamespaceFlags> PageV2<FileMetadataTemplateNamespace> CommonErrorMessage

browseTemplates

API: Internal/Beta Auth: Users

Request Response Error
FileMetadataTemplatesBrowseTemplatesRequest PageV2<FileMetadataTemplate> CommonErrorMessage

retrieve

API: Internal/Beta Auth: Users

Retrieve a single resource

Request Response Error
ResourceRetrieveRequest<FileMetadataTemplateNamespaceFlags> FileMetadataTemplateNamespace CommonErrorMessage

retrieveLatest

API: Internal/Beta Auth: Users

Request Response Error
FindByStringId FileMetadataTemplate CommonErrorMessage

retrieveProducts

API: Internal/Beta Auth: Users

Retrieve product support for all accessible providers

Request Response Error
Unit SupportByProvider<Product, FileMetadataTemplateSupport> CommonErrorMessage

This endpoint will determine all providers that which the authenticated user has access to, in the current workspace. A user has access to a product, and thus a provider, if the product is either free or if the user has been granted credits to use the product.

See also:

retrieveTemplate

API: Internal/Beta Auth: Users

Request Response Error
FileMetadataTemplateAndVersion FileMetadataTemplate CommonErrorMessage

create

API: Internal/Beta Auth: Users

Creates one or more resources

Request Response Error
BulkRequest<FileMetadataTemplateNamespace.Spec> BulkResponse<FindByStringId> CommonErrorMessage

createTemplate

API: Internal/Beta Auth: Users

Request Response Error
BulkRequest<FileMetadataTemplate> BulkResponse<FileMetadataTemplateAndVersion> CommonErrorMessage

deprecate

API: Internal/Beta Auth: Users

Request Response Error
BulkRequest<FindByStringId> BulkResponse<Unit> CommonErrorMessage

init

API: Internal/Beta Auth: Users

Request (potential) initialization of resources

Request Response Error
Unit Unit CommonErrorMessage

This request is sent by the client, if the client believes that initialization of resources might be needed. NOTE: This request might be sent even if initialization has already taken place. UCloud/Core does not check if initialization has already taken place, it simply validates the request.

updateAcl

API: Internal/Beta Auth: Users

Updates the ACL attached to a resource

Request Response Error
BulkRequest<UpdatedAcl> BulkResponse<Unit> CommonErrorMessage

Data Models

FileMetadataTemplateAndVersion

API: Internal/Beta

data class FileMetadataTemplateAndVersion(
    val id: String,
    val version: String,
)
Properties
id: String
version: String

FileMetadataTemplateNamespace

API: Internal/Beta

A Resource is the core data model used to synchronize tasks between UCloud and Provider.

data class FileMetadataTemplateNamespace(
    val id: String,
    val specification: FileMetadataTemplateNamespace.Spec,
    val createdAt: Long,
    val status: FileMetadataTemplateNamespace.Status,
    val updates: List<FileMetadataTemplateNamespace.Update>,
    val owner: ResourceOwner,
    val permissions: ResourcePermissions?,
    val providerGeneratedId: String?,
)

For more information go here.

Properties
id: String A unique identifier referencing the `Resource`

The ID is unique across a provider for a single resource type.

specification: FileMetadataTemplateNamespace.Spec
createdAt: Long Timestamp referencing when the request for creation was received by UCloud
status: FileMetadataTemplateNamespace.Status Holds the current status of the `Resource`
updates: List<FileMetadataTemplateNamespace.Update> Contains a list of updates from the provider as well as UCloud

Updates provide a way for both UCloud, and the provider to communicate to the user what is happening with their resource.

owner: ResourceOwner Contains information about the original creator of the `Resource` along with project association
permissions: ResourcePermissions? Permissions assigned to this resource

A null value indicates that permissions are not supported by this resource type.

providerGeneratedId: String?

FileMetadataTemplateNamespace.Spec

API: Internal/Beta

data class Spec(
    val name: String,
    val namespaceType: FileMetadataTemplateNamespaceType,
    val product: ProductReference,
)
Properties
name: String
namespaceType: FileMetadataTemplateNamespaceType
product: ProductReference

FileMetadataTemplateNamespace.Status

API: Internal/Beta

Describes the current state of the Resource

data class Status(
    val latestTitle: String?,
    val resolvedSupport: ResolvedSupport<Product, FileMetadataTemplateSupport>?,
    val resolvedProduct: Product?,
)

The contents of this field depends almost entirely on the specific Resource that this field is managing. Typically, this will contain information such as:

  • A state value. For example, a compute Job might be RUNNING

  • Key metrics about the resource.

  • Related resources. For example, certain Resources are bound to another Resource in a mutually exclusive way, this should be listed in the status section.

Properties
latestTitle: String?
resolvedSupport: ResolvedSupport<Product, FileMetadataTemplateSupport>?
resolvedProduct: Product? The resolved product referenced by `product`.

This attribute is not included by default unless includeProduct is specified.


FileMetadataTemplateNamespace.Update

API: Stable

Describes an update to the Resource

data class Update(
    val timestamp: Long,
    val status: String?,
)

Updates can optionally be fetched for a Resource. The updates describe how the Resource changes state over time. The current state of a Resource can typically be read from its status field. Thus, it is typically not needed to use the full update history if you only wish to know the current state of a Resource.

An update will typically contain information similar to the status field, for example:

  • A state value. For example, a compute Job might be RUNNING.

  • Change in key metrics.

  • Bindings to related Resources.

Properties
timestamp: Long A timestamp referencing when UCloud received this update
status: String? A generic text message describing the current status of the `Resource`

FileMetadataTemplateNamespaceFlags

API: Internal/Beta

data class FileMetadataTemplateNamespaceFlags(
    val includeOthers: Boolean?,
    val includeUpdates: Boolean?,
    val includeSupport: Boolean?,
    val includeProduct: Boolean?,
    val filterCreatedBy: String?,
    val filterCreatedAfter: Long?,
    val filterCreatedBefore: Long?,
    val filterProvider: String?,
    val filterProductId: String?,
    val filterProductCategory: String?,
    val filterProviderIds: String?,
    val filterIds: String?,
    val filterName: String?,
    val hideProductId: String?,
    val hideProductCategory: String?,
    val hideProvider: String?,
)
Properties
includeOthers: Boolean?
includeUpdates: Boolean?
includeSupport: Boolean?
includeProduct: Boolean? Includes `specification.resolvedProduct`
filterCreatedBy: String?
filterCreatedAfter: Long?
filterCreatedBefore: Long?
filterProvider: String?
filterProductId: String?
filterProductCategory: String?
filterProviderIds: String? Filters by the provider ID. The value is comma-separated.
filterIds: String? Filters by the resource ID. The value is comma-separated.
filterName: String?
hideProductId: String?
hideProductCategory: String?
hideProvider: String?

FileMetadataTemplateSupport

API: Internal/Beta

data class FileMetadataTemplateSupport(
    val product: ProductReference?,
    val maintenance: Maintenance?,
)
Properties
product: ProductReference?
maintenance: Maintenance?

FileMetadataTemplatesBrowseTemplatesRequest

API: Internal/Beta

The base type for requesting paginated content.

data class FileMetadataTemplatesBrowseTemplatesRequest(
    val id: String,
    val itemsPerPage: Int?,
    val next: String?,
    val consistency: PaginationRequestV2Consistency?,
    val itemsToSkip: Long?,
)

Paginated content can be requested with one of the following consistency guarantees, this greatly changes the semantics of the call:

Consistency Description
PREFER Consistency is preferred but not required. An inconsistent snapshot might be returned.
REQUIRE Consistency is required. A request will fail if consistency is no longer guaranteed.

The consistency refers to if collecting all the results via the pagination API are consistent. We consider the results to be consistent if it contains a complete view at some point in time. In practice this means that the results must contain all the items, in the correct order and without duplicates.

If you use the PREFER consistency then you may receive in-complete results that might appear out-of-order and can contain duplicate items. UCloud will still attempt to serve a snapshot which appears mostly consistent. This is helpful for user-interfaces which do not strictly depend on consistency but would still prefer something which is mostly consistent.

The results might become inconsistent if the client either takes too long, or a service instance goes down while fetching the results. UCloud attempts to keep each next token alive for at least one minute before invalidating it. This does not mean that a client must collect all results within a minute but rather that they must fetch the next page within a minute of the last page. If this is not feasible and consistency is not required then PREFER should be used.


📝 NOTE: Services are allowed to ignore extra criteria of the request if the next token is supplied. This is needed in order to provide a consistent view of the results. Clients should provide the same criterion as they paginate through the results.


Properties
id: String
itemsPerPage: Int? Requested number of items per page. Supported values: 10, 25, 50, 100, 250.
next: String? A token requesting the next page of items
consistency: PaginationRequestV2Consistency? Controls the consistency guarantees provided by the backend
itemsToSkip: Long? Items to skip ahead