About the structure of datasets, directory paths, and URLS
- About the structure of datasets, directory paths, and URLS
- Data set structure in InfiniCLOUD
- Role of each data set
- Regarding application-specific data sets
- (Reference) URL-accessible mechanism for data sets
What is a ZFS dataset?
The ZFS dataset used for InfiniCLOUD's file system has a nested structure similar to a directory.
Since each dataset can have different properties, it is a control unit for various functions such as acquisition of values in relation to capacity, creation of snapshots, reference, rollback, difference, quota, compression management, encryption, etc.
In addition, since properties are inherited by parent and child, the basic settings will be inherited from the parent, and the capacity used and other properties include the child.
These are managed in a path-like format.
DATA SET name / /backup /backup/data.ZsXw /backup/data.WtPO /files
As shown below, it is actually mounted to the directory path as follows.
| DATA SET | MOUNTPOINT |
|---|---|
| / | ./ |
| /backup | ./backup |
| /backup/data.ZsXw | ./backup/data.ZsXw |
| /backup/data.WtPO | ./backup/data.WtPO |
| /files | ./files |
Mount points reference the placement in the path.
In the above dataset structure, if there is a directory called ./backup/foo, it is just a directory named foo placed in the backup dataset, and there is no dataset /backup/foo.Creating a dataset is more expensive than creating a directory, therefore, we do not create a dataset for each directory.
If you want to create a dataset instead of a directory, it is recommended to manage the dataset resource uniquely, as it is used when you want to change the management classification.
The data set in the preceding is the generic name of the following items:
- file system
- snap shot
The data set mentioned above is the file system of the data set.
About the file system
A file system in ZFS is a dataset that can be mounted to a directory path.
What is a snapshot?
A snapshot is a function that can save an image of a dataset-file system as it appears at a specific point in time. This can be obtained in near-instantaneous timing. Even if data is written after the snapshot is created, only differential data will be recorded in the file system. As a result, the present state can be preserved in a low-cost manner.
Snapshots can be taken against the dataset's filesystem. It is necessary to do this on a dataset-by-dataset basis. If a recursive is set, it is also possible to retrieve data at the same time, including lower datasets and file systems. Note that snapshots of snapshots cannot be obtained.
There is a reference-only directory under /.zfs/snapshot. For reference, there is a reference-only directory with the same name as the snapshot name. A snapshot can only refer to files within the dataset itself. If you create a dataset called files/foo, snapshots placed in the directory under ./foo will be placed under /files/foo/.zfs/snapshot. /files/.zfs/<snapshot name>/foo does not exist.
The snapshot preserves the status of the system at that point in time; it is not a backup. It is, nonetheless, extremely beneficial for recovering from logical error or accidental deletion of the original file.
In InfiniCLOUD, only 100 snapshots can be taken per each user.
Although snapshots can be created almost instantly, deleting them to free up occupied data space will take time.
What is rollback?
Rollback is a function to rewind the file system to the point in time of the snapshot. It discards everything written after the snapshot and restores the file system to its original state.
In the case of rolling back, it is theoretically impossible to undo the rollback because all of the progress snapshots are destroyed.
It is unsupported by InfiniCLOUD at this moment, but rollback support is intended for future use and support.
What is a clone?
Cloning is a function that creates a new file system based on a snapshot.
Since only the difference data after the snapshot is retained, if the same data is included, it is possible to prevent capacity pressure.
It is unsupported by InfiniCLOUD at this moment, but rollback support is intended for future use and support.
What is file system level encryption?
This is a function that performs encryption such as AES for each file system.
This file system requires a password or key when mounted. In the event the password or key is lost, the data will be encrypted and can not be recovered.
It is unsupported by InfiniCLOUD at this moment, but rollback support is intended for future use and support.
Data set structure in InfiniCLOUD
InfiniCLOUD user's have one dataset-file system when their account is created, which is mounted on the directory.
The user's data is stored in this dataset-file system, which is mounted on a mount point with the same name.
Only the data set mapped with the mount path, which is accessed via the HTTPS (WebDAV) protocol, can be accessed through URL. Only /dav/ can be freely accessed by the general user, and the area below /backup can be used uniquely by applications.
| DATA SET | MOUNTPOINT | WebDAV URL |
|---|---|---|
| / | / | None |
| /backup | /backup | /backup *WebDAV with BASIC authentication only; cannot be viewed from a browser. The same applies to the following paths. |
| /backup/JUSTPLAYER | /backup/JUSTPLAYER | /backup/JUSTPLAYER |
| /backup/JUSTPLAYER/appname | /backup/JUSTPLAYER/appname | /backup/JUSTPLAYER/appname |
| /backup/data.ZsXw | /backup/data.ZsXW | /backup/data.ZsXW |
| /backup/data.WtPO | /backup/data.WtPO | /backup/data.WtPO |
| /files | /files | /dav/ *For BASIC authentication /v2/dav/ *Browsable from browser. |
| /someappdata | /someappdata | None *In general it should not be made |
| /someappdata2 | /someappdata2 | None *In general it should not be made |
The user's root dataset and everything below is the property of the user, and the user's used capacity means the used capacity including all subordinates (descendants) of the root dataset.
However, since dataset names do not have a leading /, the root is just for convenience and will be a null string on the API.
The maximum length of a data collection is 64 characters.
In addition, all datasets can be managed with a user ID and password (account password or app password differs depending on the authentication method), and up to 32 datasets can be created wherever you like with the REST API.
In InfiniCLOUD, file system type datasets are always mounted in a location equal to the dataset name from the root. If a dataset under the name someappdata is created, it will be mounted to the path /someappdata.
If the user were to create a directory named foo and install files, the user will be unable to construct a data set called files/foo.
Role of each data set
As previously mentioned above, the InfiniCLOUD data set's name each have their own meanings and significance.
| DATA SET name | URL | role |
|---|---|---|
| / | None | It is the parent of the user's area. With the exception of a small number of users, this area will not be used directly because the overall capacity measurement will be referred from here. |
| / files | /dav/ , /v2/dav/ | The user's general area. Browsers and standard web dub clients can both access it. User accounts created prior to April of 2015, will not this data set and are instead stored in the route data set. It is OK to construct the data set below, however, because this is typically where the user stores their data, if the user has already created a directory with the same name, it will be overwritten. Attention will be required if you are unable to do so. |
| /backup | /backup | It is used when an application provides a certain area that the user cannot readily touch. Backup data, application data storage, and other applications are just a few examples. |
| Other DATA SET Names | None | It is not recommended to create a random name. In the future, there is a possibility that a mechanism for special access to these areas may be created as requested. |
Regarding application-specific data sets
Areas that can be accessed with /dav/ or /v2/dav/ can be easily accessed by the user with a browser, so there is a risk that the user will operate them erroneously regardless of their intentions.
Depending on the application, this can lead to fatal problems, therefore, InfiniCLOUD allows you to create areas that users can not easily access with a browser.
Furthermore, because different administration and operations are handled on a data set (file system) basis, ZFS, the file system used by InfiniCLOUD, allows application develops to build data sets for each role and obtain many advantages (limit is 10 per user).
When creating a dataset (file system) in the user's /backup area, it can be created with the following rules.
- /backup/<VendorID>/<AppName>
- VendorID, AppName can be any name you desire, however it should be up to 64 characters in total length, and exclusively alphanumeric. Because symbols with spaces are not permitted, it is recommend that the VendorID and AppName be something simple.
Because it is not referable from a browser, this is not the case if the user chooses to offer that path to a general-purpose WebDAV client.
Example: /backup/JUSTPLAYER/tctool etc. - /backup/data.XXXX
- When you don't have to or don't want to put the vendor's or app's name in, use this method. To be placed in the same hierarchy as all vendors or applications, the name of the data.XXXX must be unique. You must use (unique) in the data set API's insert data set to suffix the data.XXXX so that it is generated at random by the system.
It's important to note that these data sets are not completely inaccessible to the user; they can be erased using the browser's Data Set Editor.
(Reference) URL-accessible mechanism for data sets
A data set (file system) name and a mount point have a unique association in InfiniCLOUD, however there is no such relationship between a data set and a published URL.
The region that users can see through the web interface or WebDAV is usually accessible via the path "/dav/".
| URL layer | /dav/ |
|---|---|
| URL path conversion rules (fixed) | /dav/ → /dav/ |
| Directory Path Layer | /dav/ |
| Mapper /dav/ → /files/ | |
| /files | |
| Mounting path rules (fixed) | /files → files |
| Data Set Layer | files |
"File path: /files/" is URL mapped to "URL path: /dav". "File path: /files/" is where the "Data set: files" is mounted (*There is no files data set for users created before April 2015, and a directory named /files/ is generated in the root data set).
The above URL mapper is expected to be swapped in the forthcoming mapper API, allowing for a variety of uses, such as preserving the Clone at a specific point in time. However, because this feature has numerous side effects, such as authentication cache, it is currently being considered for implementation and release.