Drives (FileCollection)¶
A FileCollection is an entrypoint to a user’s files
Rationale¶
📝 NOTE: This API follows the standard Resources API. We recommend that you have already read and understood the concepts described here.
This entrypoint allows the user to access all the files they have access to within a single project. It is important to note that a file collection is not the same as a directory! Common real-world examples of a file collection is listed below:
Name | Typical path | Comment |
---|---|---|
Home directory | /home/$username/ |
The home folder is typically the main collection for a user |
Work directory | /work/$projectId/ |
The project 'home' folder |
Scratch directory | /scratch/$projectId/ |
Temporary storage for a project |
The provider of storage manages a ‘database’ of these file collections and who they belong to. The file collections also play an important role in accounting and resource management. A file collection can have a quota attached to it and billing information is also stored in this object. Each file collection can be attached to a different product type, and as a result, can have different billing information attached to it. This is, for example, useful if a storage provider has both fast and slow tiers of storage, which is typically billed very differently.
All file collections additionally have a title. This title can be used for a user-friendly version of the folder. This
title does not have to be unique, and can with great benefit choose to not reference who it belongs to. For example,
if every user has exactly one home directory, then it would make sense to give this collection "Home"
as its title.
📝 Provider Note: This is the API exposed to end-users. See the table below for other relevant APIs.
End-User | Provider (Ingoing) | Control (Outgoing) |
---|---|---|
FileCollections |
FileCollectionsProvider |
FileCollectionsControl |
Table of Contents¶
1. Examples
Description |
---|
An example collection |
Renaming a collection |
2. Remote Procedure Calls
Name | Description |
---|---|
browse |
Browses the catalog of available resources |
retrieve |
Retrieve a single resource |
retrieveProducts |
Retrieve product support for all accessible providers |
search |
Searches the catalog of available resources |
create |
Creates one or more resources |
delete |
Deletes one or more resources |
init |
Request (potential) initialization of resources |
rename |
Request renaming of [`FileCollection`](/docs/reference/dk.sdu.cloud.file.orchestrator.api.FileCollection.md) |
updateAcl |
Updates the ACL attached to a resource |
3. Data Models
Name | Description |
---|---|
FSCollectionSupport |
Declares which `FileCollection` operations are supported for a product |
FSFileSupport |
Declares which file-level operations a product supports |
FSProductStatsSupport |
Declares which stats a given product supports |
FSSupport |
No description |
FileCollection |
A `Resource` is the core data model used to synchronize tasks between UCloud and Provider. |
FileCollection.Spec |
No description |
FileCollection.Status |
Describes the current state of the `Resource` |
FileCollection.Update |
Describes an update to the `Resource` |
FileCollectionIncludeFlags |
No description |
MemberFilesFilter |
Filter for member files. |
FileCollectionsRenameRequestItem |
No description |
Example: An example collection¶
Frequency of use | Common |
---|---|
Actors |
|
Communication Flow: Kotlin
/* In this example we will see a simple collection. This collection models the 'home' directory of a user. */
/* 📝 NOTE: Collections are identified by a unique (UCloud provided) ID */
FileCollections.retrieve.call(
ResourceRetrieveRequest(
flags = FileCollectionIncludeFlags(
filterCreatedAfter = null,
filterCreatedBefore = null,
filterCreatedBy = null,
filterIds = null,
filterMemberFiles = null,
filterProductCategory = null,
filterProductId = null,
filterProvider = null,
filterProviderIds = null,
hideProductCategory = null,
hideProductId = null,
hideProvider = null,
includeOthers = false,
includeProduct = false,
includeSupport = false,
includeUpdates = false,
),
id = "54123",
),
user
).orThrow()
/*
FileCollection(
createdAt = 1635151675465,
id = "54123",
owner = ResourceOwner(
createdBy = "user",
project = null,
),
permissions = ResourcePermissions(
myself = listOf(Permission.ADMIN),
others = emptyList(),
),
providerGeneratedId = null,
specification = FileCollection.Spec(
product = ProductReference(
category = "example-ssd",
id = "example-ssd",
provider = "example",
),
title = "Home",
),
status = FileCollection.Status(
resolvedProduct = null,
resolvedSupport = null,
),
updates = emptyList(),
)
*/
Communication Flow: Curl
# ------------------------------------------------------------------------------------------------------
# $host is the UCloud instance to contact. Example: 'http://localhost:8080' or 'https://cloud.sdu.dk'
# $accessToken is a valid access-token issued by UCloud
# ------------------------------------------------------------------------------------------------------
# In this example we will see a simple collection. This collection models the 'home' directory of a user.
# 📝 NOTE: Collections are identified by a unique (UCloud provided) ID
# Authenticated as user
curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/retrieve?includeOthers=false&includeUpdates=false&includeSupport=false&includeProduct=false&id=54123"
# {
# "id": "54123",
# "specification": {
# "title": "Home",
# "product": {
# "id": "example-ssd",
# "category": "example-ssd",
# "provider": "example"
# }
# },
# "createdAt": 1635151675465,
# "status": {
# "resolvedSupport": null,
# "resolvedProduct": null
# },
# "updates": [
# ],
# "owner": {
# "createdBy": "user",
# "project": null
# },
# "permissions": {
# "myself": [
# "ADMIN"
# ],
# "others": [
# ]
# },
# "providerGeneratedId": null
# }
Communication Flow: Visual
Example: Renaming a collection¶
Frequency of use | Common |
---|---|
Actors |
|
Communication Flow: Kotlin
/* In this example, we will see how a user can rename one of their collections. */
/* 📝 NOTE: Renaming must be supported by the provider */
FileCollections.retrieveProducts.call(
Unit,
user
).orThrow()
/*
SupportByProvider(
productsByProvider = mapOf("ucloud" to listOf(ResolvedSupport(
product = Product.Storage(
allowAllocationRequestsFrom = AllocationRequestsGroup.ALL,
category = ProductCategoryId(
id = "example-ssd",
name = "example-ssd",
provider = "example",
),
chargeType = ChargeType.DIFFERENTIAL_QUOTA,
description = "Fast storage",
freeToUse = false,
hiddenInGrantApplications = false,
name = "example-ssd",
pricePerUnit = 1,
priority = 0,
productType = ProductType.STORAGE,
unitOfPrice = ProductPriceUnit.PER_UNIT,
version = 1,
balance = null,
id = "example-ssd",
maxUsableBalance = null,
),
support = FSSupport(
collection = FSCollectionSupport(
aclModifiable = null,
usersCanCreate = true,
usersCanDelete = true,
usersCanRename = true,
),
files = FSFileSupport(
aclModifiable = false,
isReadOnly = false,
searchSupported = true,
sharesSupported = false,
streamingSearchSupported = false,
trashSupported = false,
),
maintenance = null,
product = ProductReference(
category = "example-ssd",
id = "example-ssd",
provider = "example",
),
stats = FSProductStatsSupport(
accessedAt = null,
createdAt = null,
modifiedAt = null,
sizeInBytes = null,
sizeIncludingChildrenInBytes = null,
unixGroup = null,
unixOwner = null,
unixPermissions = null,
),
),
))),
)
*/
/* As we can see, the provider does support the rename operation. We now look at our collections. */
FileCollections.browse.call(
ResourceBrowseRequest(
consistency = null,
flags = FileCollectionIncludeFlags(
filterCreatedAfter = null,
filterCreatedBefore = null,
filterCreatedBy = null,
filterIds = null,
filterMemberFiles = null,
filterProductCategory = null,
filterProductId = null,
filterProvider = null,
filterProviderIds = null,
hideProductCategory = null,
hideProductId = null,
hideProvider = null,
includeOthers = false,
includeProduct = false,
includeSupport = false,
includeUpdates = false,
),
itemsPerPage = null,
itemsToSkip = null,
next = null,
sortBy = null,
sortDirection = null,
),
user
).orThrow()
/*
PageV2(
items = listOf(FileCollection(
createdAt = 1635151675465,
id = "54123",
owner = ResourceOwner(
createdBy = "user",
project = null,
),
permissions = ResourcePermissions(
myself = listOf(Permission.ADMIN),
others = emptyList(),
),
providerGeneratedId = null,
specification = FileCollection.Spec(
product = ProductReference(
category = "example-ssd",
id = "example-ssd",
provider = "example",
),
title = "Home",
),
status = FileCollection.Status(
resolvedProduct = null,
resolvedSupport = null,
),
updates = emptyList(),
)),
itemsPerPage = 50,
next = null,
)
*/
/* Using the unique ID, we can rename the collection */
FileCollections.rename.call(
bulkRequestOf(FileCollectionsRenameRequestItem(
id = "54123",
newTitle = "My Awesome Drive",
)),
user
).orThrow()
/*
Unit
*/
/* The new title is observed when we browse the collections one more time */
FileCollections.browse.call(
ResourceBrowseRequest(
consistency = null,
flags = FileCollectionIncludeFlags(
filterCreatedAfter = null,
filterCreatedBefore = null,
filterCreatedBy = null,
filterIds = null,
filterMemberFiles = null,
filterProductCategory = null,
filterProductId = null,
filterProvider = null,
filterProviderIds = null,
hideProductCategory = null,
hideProductId = null,
hideProvider = null,
includeOthers = false,
includeProduct = false,
includeSupport = false,
includeUpdates = false,
),
itemsPerPage = null,
itemsToSkip = null,
next = null,
sortBy = null,
sortDirection = null,
),
user
).orThrow()
/*
PageV2(
items = listOf(FileCollection(
createdAt = 1635151675465,
id = "54123",
owner = ResourceOwner(
createdBy = "user",
project = null,
),
permissions = ResourcePermissions(
myself = listOf(Permission.ADMIN),
others = emptyList(),
),
providerGeneratedId = null,
specification = FileCollection.Spec(
product = ProductReference(
category = "example-ssd",
id = "example-ssd",
provider = "example",
),
title = "My Awesome Drive",
),
status = FileCollection.Status(
resolvedProduct = null,
resolvedSupport = null,
),
updates = emptyList(),
)),
itemsPerPage = 50,
next = null,
)
*/
Communication Flow: Curl
# ------------------------------------------------------------------------------------------------------
# $host is the UCloud instance to contact. Example: 'http://localhost:8080' or 'https://cloud.sdu.dk'
# $accessToken is a valid access-token issued by UCloud
# ------------------------------------------------------------------------------------------------------
# In this example, we will see how a user can rename one of their collections.
# 📝 NOTE: Renaming must be supported by the provider
# Authenticated as user
curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/retrieveProducts"
# {
# "productsByProvider": {
# "ucloud": [
# {
# "product": {
# "balance": null,
# "maxUsableBalance": null,
# "name": "example-ssd",
# "pricePerUnit": 1,
# "category": {
# "name": "example-ssd",
# "provider": "example"
# },
# "description": "Fast storage",
# "priority": 0,
# "version": 1,
# "freeToUse": false,
# "allowAllocationRequestsFrom": "ALL",
# "unitOfPrice": "PER_UNIT",
# "chargeType": "DIFFERENTIAL_QUOTA",
# "hiddenInGrantApplications": false,
# "productType": "STORAGE"
# },
# "support": {
# "product": {
# "id": "example-ssd",
# "category": "example-ssd",
# "provider": "example"
# },
# "stats": {
# "sizeInBytes": null,
# "sizeIncludingChildrenInBytes": null,
# "modifiedAt": null,
# "createdAt": null,
# "accessedAt": null,
# "unixPermissions": null,
# "unixOwner": null,
# "unixGroup": null
# },
# "collection": {
# "aclModifiable": null,
# "usersCanCreate": true,
# "usersCanDelete": true,
# "usersCanRename": true
# },
# "files": {
# "aclModifiable": false,
# "trashSupported": false,
# "isReadOnly": false,
# "searchSupported": true,
# "streamingSearchSupported": false,
# "sharesSupported": false
# },
# "maintenance": null
# }
# }
# ]
# }
# }
# As we can see, the provider does support the rename operation. We now look at our collections.
curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/browse?includeOthers=false&includeUpdates=false&includeSupport=false&includeProduct=false"
# {
# "itemsPerPage": 50,
# "items": [
# {
# "id": "54123",
# "specification": {
# "title": "Home",
# "product": {
# "id": "example-ssd",
# "category": "example-ssd",
# "provider": "example"
# }
# },
# "createdAt": 1635151675465,
# "status": {
# "resolvedSupport": null,
# "resolvedProduct": null
# },
# "updates": [
# ],
# "owner": {
# "createdBy": "user",
# "project": null
# },
# "permissions": {
# "myself": [
# "ADMIN"
# ],
# "others": [
# ]
# },
# "providerGeneratedId": null
# }
# ],
# "next": null
# }
# Using the unique ID, we can rename the collection
curl -XPOST -H "Authorization: Bearer $accessToken" -H "Content-Type: content-type: application/json; charset=utf-8" "$host/api/files/collections/rename" -d '{
"items": [
{
"id": "54123",
"newTitle": "My Awesome Drive"
}
]
}'
# {
# }
# The new title is observed when we browse the collections one more time
curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/browse?includeOthers=false&includeUpdates=false&includeSupport=false&includeProduct=false"
# {
# "itemsPerPage": 50,
# "items": [
# {
# "id": "54123",
# "specification": {
# "title": "My Awesome Drive",
# "product": {
# "id": "example-ssd",
# "category": "example-ssd",
# "provider": "example"
# }
# },
# "createdAt": 1635151675465,
# "status": {
# "resolvedSupport": null,
# "resolvedProduct": null
# },
# "updates": [
# ],
# "owner": {
# "createdBy": "user",
# "project": null
# },
# "permissions": {
# "myself": [
# "ADMIN"
# ],
# "others": [
# ]
# },
# "providerGeneratedId": null
# }
# ],
# "next": null
# }
Communication Flow: Visual
Remote Procedure Calls¶
browse
¶
Browses the catalog of available resources
Request | Response | Error |
---|---|---|
ResourceBrowseRequest<FileCollectionIncludeFlags> |
PageV2<FileCollection> |
CommonErrorMessage |
retrieve
¶
Retrieve a single resource
Request | Response | Error |
---|---|---|
ResourceRetrieveRequest<FileCollectionIncludeFlags> |
FileCollection |
CommonErrorMessage |
retrieveProducts
¶
Retrieve product support for all accessible providers
Request | Response | Error |
---|---|---|
Unit |
SupportByProvider<Product.Storage, FSSupport> |
CommonErrorMessage |
This endpoint will determine all providers that which the authenticated user has access to, in the current workspace. A user has access to a product, and thus a provider, if the product is either free or if the user has been granted credits to use the product.
See also:
search
¶
Searches the catalog of available resources
Request | Response | Error |
---|---|---|
ResourceSearchRequest<FileCollectionIncludeFlags> |
PageV2<FileCollection> |
CommonErrorMessage |
create
¶
Creates one or more resources
Request | Response | Error |
---|---|---|
BulkRequest<FileCollection.Spec> |
BulkResponse<FindByStringId> |
CommonErrorMessage |
delete
¶
Deletes one or more resources
Request | Response | Error |
---|---|---|
BulkRequest<FindByStringId> |
BulkResponse<Unit> |
CommonErrorMessage |
init
¶
Request (potential) initialization of resources
Request | Response | Error |
---|---|---|
Unit |
Unit |
CommonErrorMessage |
This request is sent by the client, if the client believes that initialization of resources might be needed. NOTE: This request might be sent even if initialization has already taken place. UCloud/Core does not check if initialization has already taken place, it simply validates the request.
rename
¶
Request renaming of FileCollection
Request | Response | Error |
---|---|---|
BulkRequest<FileCollectionsRenameRequestItem> |
Unit |
CommonErrorMessage |
updateAcl
¶
Updates the ACL attached to a resource
Request | Response | Error |
---|---|---|
BulkRequest<UpdatedAcl> |
BulkResponse<Unit> |
CommonErrorMessage |
Data Models¶
FSCollectionSupport
¶
Declares which FileCollection
operations are supported for a product
data class FSCollectionSupport(
val aclModifiable: Boolean?,
val usersCanCreate: Boolean?,
val usersCanDelete: Boolean?,
val usersCanRename: Boolean?,
)
FSFileSupport
¶
Declares which file-level operations a product supports
data class FSFileSupport(
val aclModifiable: Boolean?,
val trashSupported: Boolean?,
val isReadOnly: Boolean?,
val searchSupported: Boolean?,
val streamingSearchSupported: Boolean?,
val sharesSupported: Boolean?,
)
Properties
aclModifiable
: Boolean?
Boolean?
trashSupported
: Boolean?
Boolean?
isReadOnly
: Boolean?
Boolean?
searchSupported
: Boolean?
Declares support for the normal search endpoint
Boolean?
NOTE(Dan, 01/09/2022): For backwards compatibility, this is true by default, however, this will likely change to false in a later release. Providers should explicitly declare support for this endpoint for the time being.
streamingSearchSupported
: Boolean?
Declares support for the streamingSearch endpoint
Boolean?
sharesSupported
: Boolean?
Boolean?
FSProductStatsSupport
¶
Declares which stats a given product supports
data class FSProductStatsSupport(
val sizeInBytes: Boolean?,
val sizeIncludingChildrenInBytes: Boolean?,
val modifiedAt: Boolean?,
val createdAt: Boolean?,
val accessedAt: Boolean?,
val unixPermissions: Boolean?,
val unixOwner: Boolean?,
val unixGroup: Boolean?,
)
FSSupport
¶
data class FSSupport(
val product: ProductReference,
val stats: FSProductStatsSupport?,
val collection: FSCollectionSupport?,
val files: FSFileSupport?,
val maintenance: Maintenance?,
)
Properties
product
: ProductReference
ProductReference
stats
: FSProductStatsSupport?
FSProductStatsSupport?
collection
: FSCollectionSupport?
FSCollectionSupport?
files
: FSFileSupport?
FSFileSupport?
maintenance
: Maintenance?
Maintenance?
FileCollection
¶
A Resource
is the core data model used to synchronize tasks between UCloud and Provider.
data class FileCollection(
val id: String,
val specification: FileCollection.Spec,
val createdAt: Long,
val status: FileCollection.Status,
val updates: List<FileCollection.Update>,
val owner: ResourceOwner,
val permissions: ResourcePermissions?,
val providerGeneratedId: String?,
)
For more information go here.
Properties
id
: String
A unique identifier referencing the `Resource`
String
The ID is unique across a provider for a single resource type.
specification
: FileCollection.Spec
FileCollection.Spec
createdAt
: Long
Timestamp referencing when the request for creation was received by UCloud
Long
status
: FileCollection.Status
Holds the current status of the `Resource`
FileCollection.Status
updates
: List<FileCollection.Update>
Contains a list of updates from the provider as well as UCloud
List<FileCollection.Update>
Updates provide a way for both UCloud, and the provider to communicate to the user what is happening with their resource.
owner
: ResourceOwner
Contains information about the original creator of the `Resource` along with project association
ResourceOwner
permissions
: ResourcePermissions?
Permissions assigned to this resource
ResourcePermissions?
A null value indicates that permissions are not supported by this resource type.
providerGeneratedId
: String?
String?
FileCollection.Spec
¶
data class Spec(
val title: String,
val product: ProductReference,
)
Properties
title
: String
String
product
: ProductReference
A reference to the product which backs this `Resource`
ProductReference
FileCollection.Status
¶
Describes the current state of the Resource
data class Status(
val resolvedSupport: ResolvedSupport<Product.Storage, FSSupport>?,
val resolvedProduct: Product.Storage?,
)
The contents of this field depends almost entirely on the specific Resource
that this field is managing. Typically,
this will contain information such as:
A state value. For example, a compute
Job
might beRUNNING
Key metrics about the resource.
Related resources. For example, certain
Resource
s are bound to anotherResource
in a mutually exclusive way, this should be listed in thestatus
section.
Properties
resolvedSupport
: ResolvedSupport<Product.Storage, FSSupport>?
ResolvedSupport<Product.Storage, FSSupport>?
resolvedProduct
: Product.Storage?
The resolved product referenced by `product`.
Product.Storage?
This attribute is not included by default unless includeProduct
is specified.
FileCollection.Update
¶
Describes an update to the Resource
data class Update(
val timestamp: Long,
val status: String?,
)
Updates can optionally be fetched for a Resource
. The updates describe how the Resource
changes state over time.
The current state of a Resource
can typically be read from its status
field. Thus, it is typically not needed to
use the full update history if you only wish to know the current state of a Resource
.
An update will typically contain information similar to the status
field, for example:
A state value. For example, a compute
Job
might beRUNNING
.Change in key metrics.
Bindings to related
Resource
s.
FileCollectionIncludeFlags
¶
data class FileCollectionIncludeFlags(
val filterMemberFiles: MemberFilesFilter?,
val includeOthers: Boolean?,
val includeUpdates: Boolean?,
val includeSupport: Boolean?,
val includeProduct: Boolean?,
val filterCreatedBy: String?,
val filterCreatedAfter: Long?,
val filterCreatedBefore: Long?,
val filterProvider: String?,
val filterProductId: String?,
val filterProductCategory: String?,
val filterProviderIds: String?,
val filterIds: String?,
val hideProductId: String?,
val hideProductCategory: String?,
val hideProvider: String?,
)
Properties
filterMemberFiles
: MemberFilesFilter?
MemberFilesFilter?
includeOthers
: Boolean?
Boolean?
includeUpdates
: Boolean?
Boolean?
includeSupport
: Boolean?
Boolean?
includeProduct
: Boolean?
Includes `specification.resolvedProduct`
Boolean?
filterCreatedBy
: String?
String?
filterCreatedAfter
: Long?
Long?
filterCreatedBefore
: Long?
Long?
filterProvider
: String?
String?
filterProductId
: String?
String?
filterProductCategory
: String?
String?
filterProviderIds
: String?
Filters by the provider ID. The value is comma-separated.
String?
filterIds
: String?
Filters by the resource ID. The value is comma-separated.
String?
hideProductId
: String?
String?
hideProductCategory
: String?
String?
hideProvider
: String?
String?
MemberFilesFilter
¶
Filter for member files.
enum class MemberFilesFilter {
SHOW_ONLY_MINE,
SHOW_ONLY_MEMBER_FILES,
DONT_FILTER_COLLECTIONS,
}
A member files collection must use the following format to be recognized: “Member Files: $username”
Properties
SHOW_ONLY_MINE
Shows only the requesting user's personal member file along with all other collections
SHOW_ONLY_MEMBER_FILES
Shows only the member file collections and hides all others
DONT_FILTER_COLLECTIONS
Applies no filter and shows both normal collections and member files
FileCollectionsRenameRequestItem
¶
data class FileCollectionsRenameRequestItem(
val id: String,
val newTitle: String,
)