Plugin authoring guide

TurboDL is a standalone download engine and an optional plugin platform. This directory is the home for everything a plugin author needs:

• Plugin authoring guide — this file: concepts + a step-by-step guide.
• Development Convention — the official compatibility rulebook (stable API, versioning, naming, safety). Read this before publishing.
• Plugin Market — how to publish/discover plugins via GitHub topics + a turbodl-plugin.json manifest (schema).

Runnable examples live in the demo module: ./gradlew :demo:run --args="1" (Kotlin plugin), "2" (bootstrap), "3" (shim adapter).

Philosophy#

Three ideas drive the design:

1. Core works alone. turbodl-core is a complete, standalone multi-threaded download engine with zero plugin dependency. Plugins are strictly additive; if you never load one, nothing changes.

1. Everything is a plugin; the kernel is only mechanism. The runtime kernel (turbo-plugin-runtime) knows nothing about HTTP, HLS, or any protocol. It provides only mechanism: lifecycle, cleanup (disposer), a service registry with dependency resolution, a type-safe event bus, an extension-point registry, a version handshake, and diagnostics. The Kotlin loader, the HTTP backend, the HLS backend — all of them are ordinary plugins. Even a future JS runtime would be just a plugin implementing PluginLoaderProvider.

1. Hybrid A+B. (A) Core ships a built-in HTTP backend so it is useful out of the box. (B) A plugin backend can override the built-in one or add new protocols through the extension registry — without core ever depending on the runtime. Direction of dependency is strict: runtime → core, never the reverse.

The result: a small, stable contract surface that third parties can build against, with a compatibility policy (the Convention) so a future breaking core release fails loudly instead of silently corrupting plugins.

The building blocks#

Built-in extension points (dev.turbodl.plugin.runtime.ext.ExtensionPoints):

• DOWNLOAD_BACKEND — add/override a protocol (DownloadBackend)
• PROTOCOL_HANDLER — declare which protocols/schemes you handle (ProtocolClaim; indexed at load time by host.protocols — a declaration only, it routes nothing)
• LINK_PARSER — turn a raw link into DownloadRequests (LinkParser)
• TASK_PRE_HOOK — rewrite a request before submit (TaskPreHook)
• TASK_POST_HOOK — react after a task finishes (TaskPostHook)

The only extension point the kernel knows by name is PluginLoaderProvider (loaders).

Authoring a plugin#

1. Depend on the contracts#

dependencies {
    implementation("dev.turbodl:turbodl-core:<version>")
    implementation("dev.turbodl:turbo-plugin-runtime:<version>")
}

2. Implement Plugin#

import dev.turbodl.core.ApiVersion
import dev.turbodl.plugin.runtime.Plugin
import dev.turbodl.plugin.runtime.PluginContext

class MyPlugin : Plugin {
    override val id = "adapter.acme"                 // unique, dot-separated, vendor-qualified
    override val name = "ACME Adapter"
    override val requiredApiVersion = ApiVersion(1, 0, 0)   // lowest API you actually use

    override fun onLoad(context: PluginContext) {
        // register services / events / extensions here — all auto-cleaned on unload
    }
}

3. Register capabilities in onLoad#

// A service other plugins may depend on:
context.registerService(id, myService)

// Observe engine events (auto-unsubscribed on unload):
context.onEvent { event -> /* ... */ }

// Rewrite requests before they run:
context.interceptRequest { req -> req.copy(headers = req.headers + ("X-Trace" to "1")) }

// Provide an extension-point implementation (higher priority wins for "the" impl):
context.registerExtension(ExtensionPoints.DOWNLOAD_BACKEND, myBackend, priority = 100)

// Anything else that needs teardown:
context.disposer.register { myThreadPool.shutdown() }

Everything registered through context is torn down automatically when the plugin is uninstalled. If onLoad throws, partial registrations are rolled back for you.

4. Writing a DownloadBackend#

A backend owns the protocol; the engine owns state, events, merging and integrity. Rules (full list in Convention §7):

• supports(request) — cheap, side-effect-free, conservative.
• Honor context.isActive() and coroutine cancellation.
• context.reportTotalSize(total) once known (-1 if unknown), context.reportProgress(...) as you go, context.throttle(bytes) to respect the global speed limit.
• Write parts into context.workDir, return them in order as BackendResult.orderedParts.
• On unrecoverable/out-of-scope input, throw — never emit a corrupt file.
• Need core's HTTP transport policy (proxy/DNS/TLS)? Use TurboHttpClients.create(config); never touch core internals.

See turbo-plugin-hls for a complete, non-trivial backend (playlist parsing, AES-128, byte-range, explicit rejection of unsupported constructs).

5. Install it#

val host = PluginHost()
host.install(MyPlugin())

// Route downloads through plugin backends (falls back to built-in HTTP when none match):
val client = TurboClient(config)
client.backendResolver = BackendRegistry(host.extensions)

Or use the bootstrap convenience:

val boot = TurboBootstrap.create(extraPlugins = listOf(MyPlugin()))
val id = boot.client.submit(DownloadRequest(url, dest))

6. Verify with diagnostics#

println(host.diagnostics().render())
// Lists plugins + states (LOADED / WAITING / FAILED / INCOMPATIBLE / UNLOADED),
// extension points, services, listener counts.

If your plugin shows INCOMPATIBLE, the running TurboDL API does not satisfy your requiredApiVersion — check the handshake (Convention §2).

7. Publish#

Follow the Plugin Market steps: add turbodl-plugin.json, tag the repo with turbodl-plugin + a category + capabilities, publish an artifact, cut a release.

Declaring the protocols you support#

A plugin that handles protocols should say so, once, in a form the host can index at load time. That is what PROTOCOL_HANDLER plus the manifest protocols field are for — and it is why the ecosystem is "one plugin, many protocols": a single magnet/BT/eD2K plugin declares all three instead of shipping three plugins.

class P2pPlugin : Plugin {
    override val id = "backend.p2p"

    override fun onLoad(context: PluginContext) {
        // 1. The real routing registration — what actually handles a request.
        context.registerExtension(ExtensionPoints.DOWNLOAD_BACKEND, P2pBackend(), priority = 200)

        // 2. The declaration — one claim per protocol, indexed by the host at load time.
        //    (Same priority as the backend so the index and the router order identically.)
        listOf("magnet", "bt", "ed2k").forEach { scheme ->
            context.registerExtension(
                ExtensionPoints.PROTOCOL_HANDLER,
                ProtocolClaim(scheme = scheme, pluginId = id, priority = 200, label = "aria2-adapter"),
                priority = 200,
            )
        }
    }
}

// turbodl-plugin.json — the same names, so the market and the host agree
"protocols": ["magnet", "bt", "ed2k"],

What the declaration buys you:

• Load-time answers. host.protocols.resolve("magnet:?xt=urn:btih:...") finds the claimant without running a download; claimsFor("magnet") lists every claimant, highest priority first; allClaims() drives diagnostics and listings.
• Market filtering. The manifest protocols array lets a market — or a user reading a manifest — filter "which plugins handle ftp?" before installing anything, and host.protocols.conflicts() reports when two installed plugins declare the same protocol.
• One plugin, many protocols. There is no one-plugin-per-protocol rule: declaring a whole family in one plugin is the expected shape (fewer installs, fewer things to break).

What it does not do (read this twice):

• It does not route. Routing stays with the DownloadBackend.supports predicate. A claim with no matching predicate changes nothing about what downloads your URL.
• It grants no permission. A declaration is a label for the market and diagnostics — not a sandbox boundary, not an authorization to touch the network, the filesystem or user data. "I declared protocol X" is never license to do anything (Convention §12).

Names must be lowercase and start with a letter (^[a-z][a-z0-9+.-]{0,31}$); the hard rules, including conflict handling, are in Convention §12.

Gotcha worth knowing: a protocol name is not automatically a URL scheme. HLS declares hls, but an .m3u8 link is an http(s) URL — so resolve("https://x/y.m3u8") will never return the HLS claim, and HLS routes .m3u8 through its backend predicate instead. Declare the name the ecosystem knows the protocol by, and let the predicate own URL matching.

Shim adapters#

A "shim" wraps an external downloader/SDK and exposes it to TurboDL through LinkParser + DownloadBackend, without TurboDL knowing anything about that system. Start from the template in demo/.../Example3ShimAdapter.kt — replace the placeholder ExternalDownloader with a real SDK.

Writing plugins in JavaScript#

JavaScript is a delivered option, not a reserved one. It ships as a separate plugin — turbo-plugin-js — that implements PluginLoaderProvider, so the core and the Kotlin loader stay unaware of JS and no JS engine is pulled into the kernel unless you add that module on purpose.

val boot = TurboBootstrap.create(
    config = TurboConfig(maxConnectionsPerTask = 16),
    extraPlugins = listOf(HlsPlugin(), JsPluginLoaderPlugin()),
)

// Anything the JS loader owns is routed by source kind:
boot.host.loadSource(PluginSource(kind = "js", uri = "/opt/turbodl-plugins/acme-parser.js"))

A script gets a host/plugin ABI and no more: it computes URLs, headers and signatures, reads and writes its own sandbox directory, and logs. It never touches the download data stream and cannot register a DownloadBackend — that boundary is deliberate, and it is what keeps a script from becoming a bandwidth bottleneck. The full ABI, the lifecycle, and the exact list of what is refused are in turbo-plugin-js/README.md.