periodic link downloads

InterDiode can download an arbitrary URL. This URL will downloaded with the chosen frequency and replicated to the internal network.Once downloaded, the resource is transferred to the internal network and made available to download. Request cookies, header and body can be specified and will be used for the request. By default, SSL verification is enforced but can be disabled. Each newer download overwrite the previous one and download link on the internal network is unmodified across updates.

Creation

Updating data

By default, data update must be triggered manually and the retrieved data will be automatically transferred to the internal network at the end of this synchronization. In the settings panel, it is also possible to schedule this synchronization to be triggered automatically. The synchronization can be scheduled from every minute to once a year, including very fine-grained control to choose precisely at which minute or hour this synchronization will be triggered.

Note

Each worker can only process one synchronization at a time. If many projects need to be synchronized frequently, the number of workers must be adapted accordingly.

Access Control

Several permission levels are available and can be assigned to users individually or by groups. The accesses granted by default to anonymous or simply authenticated users can also be selected.

• no access: has no access at all..

• reader: can only read pages..

• admin: can change settings and access controls..

Only local administrators can change access controls, and default permissions for anonymous and simply authenticated users are transferred to the red side only during the first synchronization.

User preferences

Each user can set their own preferences, like pining it as a favorite source or creating a authentication token specific to it.

Resolve transfer issues

If a problem occurred during a transfer and both sides are out of sync, it is possible to force a resynchronization by sending all or part of the data again.

Deletion

If you want to delete this source, you must delete on both sides, starting with the black side to avoid any risk of data transfer to the red side after deletion. Once deleted, the source will no longer be available, and the related data will be removed during background cleanup tasks.

File integrity verification

Each file is stored along with its SHA256 digest, allowing to verify the integrity of the file and to detect any changes or corruption.

When you download a file, you can check its digest against the expected value to ensure that the file has not been tampered with or corrupted during transfer. This digest is provided in the HTTP headers, both Content-Digest (following the RFC 9530) and Digest. This can be done using various tools, such as command-line utilities or programming libraries that support digest verification. The digest is calculated using the SHA-256 algorithm and is encoded in Base64 format.

curl -L -v -I [URL]
# the -I option retrieves only the headers, and the -v option shows the details of the request and response, including the headers
Digest: sha-256=VqiR0u1OsX0ZGPCDVG9mlRjei0DC76I6r1+FXEhmuIs=
Content-Digest: sha-256=:VqiR0u1OsX0ZGPCDVG9mlRjei0DC76I6r1+FXEhmuIs=:

Propositions

A user with the appropriate permission can make a proposition, which must be validated by an administrator.