internaluploader: server + docs

One-way upload server: authentik SSO pairing (QR + app-initiated deep
link), hashed device tokens, streamed sha256-verified uploads into the
metube directory as <sha256>.<ext>.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-28 23:17:25 +01:00
committed by Michal
commit d8a606ce46
47 changed files with 4196 additions and 0 deletions

3
.gitignore vendored Normal file
View File

@@ -0,0 +1,3 @@
thelab-kubernetes-pulumi/
node_modules/
dist/

71
README.md Normal file
View File

@@ -0,0 +1,71 @@
# internaluploader
One-way file drop: an Android app that uploads files into the metube download
directory on nas001 (`/volume1/ss/data`). There is no way to browse, list or
download anything through it — uploads only.
Every file is stored as `<sha256-of-content>.<original extension>`, so names
carry nothing, re-uploads are idempotent, and a retried upload can be verified
server-side before it is accepted.
## Parts
- `server/` — Node 22 / TypeScript. Runs as a docker container **on nas001**
(Pulumi docker-over-ssh, same pattern as seaweedfs), bind-mounting
`/volume1/ss/data` at `/downloads`. Fronted by the cluster's Cilium ingress
at `https://puploader.ad.itaz.eu` (cert-manager LE cert) and exposed to the
tailnet as its own Tailscale device `puploader` (userspace proxy, SNI
passthrough) — reachable from anywhere on the tailnet, nothing public.
- `android/` — Kotlin app "InternalUploader". No settings screen, no server
address anywhere in the UI. The server URL is baked into the build and can
be replaced by whatever a pairing QR carries.
## Auth model
Browser SSO (authentik at `sso.ad.itaz.eu`) is the only identity check; the
app itself never sees a password.
1. **QR pairing** — open `https://puploader.ad.itaz.eu` in a browser, sign in
through authentik, and the page shows a QR code. Scan it with the app and
the device is paired. The QR carries a single-use pairing code (5 min TTL)
plus the server URL.
2. **In-app SSO (optional path)** — the app opens
`https://puploader.ad.itaz.eu/pair/app` in a Custom Tab; after the same
authentik login the server redirects to `internaluploader://pair?c=<code>`
and the app finishes pairing with the same exchange.
Pairing exchanges the one-time code for a long-lived **device token** (256-bit
random, stored hashed on the server, `Authorization: Bearer` on every upload).
## API (server ⇄ app contract)
QR / deep-link payload (JSON): `{"v":1,"u":"https://puploader.ad.itaz.eu","c":"<code>"}`
| Endpoint | Auth | Behaviour |
|---|---|---|
| `GET /healthz` | none | 200 `ok` |
| `GET /` | browser SSO | pairing page with QR |
| `GET /pair/app` | browser SSO | 302 → `internaluploader://pair?c=<code>` |
| `POST /api/pair` `{"code":"…"}` | pairing code | 200 `{"token":"…","name":"<user>"}`; 410 expired/used |
| `HEAD /api/object/<sha256>.<ext>` | bearer | 204 exists / 404 not — lets a retry skip the bytes |
| `PUT /api/upload?name=<sha256>.<ext>` | bearer | body = raw bytes, streamed to a temp file **on the same filesystem**, hash verified, then renamed into `/downloads`. 201 created, 200 already existed, 409 hash mismatch, 413 too large |
One-way by construction: the only read endpoint takes a full content hash the
caller must already possess.
## Building
- Server image: `server/build.sh` (podman build + push to
`mysources.co.uk/michal/internaluploader-server:<date>-<sha>`).
- APK: `cd android && JAVA_HOME=/usr/lib/jvm/java-21-openjdk ./gradlew assembleRelease`
(release signing reads `~/.internaluploader/signing.properties`; see
`android/README.md`). Publish to the private F-Droid repo with
`thelab-kubernetes-pulumi/scripts/fdroid/publish.py add <apk>`.
## Deployment
Declared in `thelab-kubernetes-pulumi` — `deployments/puploader/` plus an
authentik OIDC app (`deployments/authentik/apps/puploader.ts`), a
`k8s-deployments:puploader` config block, and a `puploader` entry in the
worker0 tailscale exposures (two-step: first apply registers the device, then
set `deviceIp` for the Cloudflare A record).

6
android/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
.gradle/
build/
local.properties
*.keystore
*.p12
.kotlin/

52
android/README.md Normal file
View File

@@ -0,0 +1,52 @@
# InternalUploader — Android app
Native Kotlin app. Pair once (QR from the browser pairing page, or in-app SSO
via a Custom Tab), then share anything to it — files upload in the background
as `<sha256>.<ext>` and failed uploads retry until they land. No settings
screen; the server address is baked in (`BuildConfig.DEFAULT_SERVER_URL`) and
can be replaced only by what a pairing QR carries.
## Building
```sh
JAVA_HOME=~/.jdks/jdk17 ./gradlew assembleRelease
```
- Needs an Android SDK at the path in `local.properties` (not committed;
`sdk.dir=$HOME/Android/Sdk`), platform android-36.
- Gradle 8.14.3 (wrapper), AGP 8.11.1, Kotlin 2.1.21. The Gradle daemon needs
JDK 17–24 — the system `java` (25) is too new, hence the explicit
`JAVA_HOME`.
- Output: `app/build/outputs/apk/release/app-release.apk`.
## Release signing
`app/build.gradle.kts` reads `~/.internaluploader/signing.properties`:
```properties
storeFile=/home/<you>/.internaluploader/release.p12
storePassword=…
keyAlias=internaluploader
keyPassword=…
```
The PKCS12 keystore was generated with:
```sh
keytool -genkeypair -keystore ~/.internaluploader/release.p12 -storetype PKCS12 \
-alias internaluploader -keyalg RSA -keysize 4096 -validity 10000
```
The keystore never enters the repo. If `signing.properties` is missing, the
release build is unsigned (fine for compile checks, not for F-Droid). Keep the
keystore backed up: F-Droid clients pin the app signature, so a lost key means
uninstall/reinstall on every device.
## Publishing to the private F-Droid repo
```sh
thelab-kubernetes-pulumi/scripts/fdroid/publish.py add app/build/outputs/apk/release/app-release.apk
```
Bump `versionCode`/`versionName` in `app/build.gradle.kts` for every published
build — F-Droid only offers updates on a higher `versionCode`.

View File

@@ -0,0 +1,87 @@
import java.util.Properties
plugins {
id("com.android.application")
id("org.jetbrains.kotlin.android")
id("com.google.devtools.ksp")
}
// Release signing is read from ~/.internaluploader/signing.properties so the
// keystore never enters the repo. Absent that file, release builds are simply
// unsigned (still useful for CI compile checks).
val signingProps = Properties().apply {
val f = file("${System.getProperty("user.home")}/.internaluploader/signing.properties")
if (f.exists()) f.inputStream().use { load(it) }
}
android {
namespace = "eu.itaz.internaluploader"
compileSdk = 36
defaultConfig {
applicationId = "eu.itaz.internaluploader"
minSdk = 26
targetSdk = 35
versionCode = 1
versionName = "1.0.0"
// The one place the server address exists. Never rendered in the UI;
// a pairing QR can carry a different origin and overrides this.
buildConfigField("String", "DEFAULT_SERVER_URL", "\"https://puploader.ad.itaz.eu\"")
}
signingConfigs {
if (signingProps.getProperty("storeFile") != null) {
create("release") {
storeFile = file(signingProps.getProperty("storeFile"))
storePassword = signingProps.getProperty("storePassword")
keyAlias = signingProps.getProperty("keyAlias")
keyPassword = signingProps.getProperty("keyPassword")
}
}
}
buildTypes {
release {
isMinifyEnabled = true
isShrinkResources = true
proguardFiles(getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro")
signingConfig = signingConfigs.findByName("release")
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
kotlinOptions {
jvmTarget = "17"
}
buildFeatures {
viewBinding = true
buildConfig = true
}
}
dependencies {
implementation("androidx.core:core-ktx:1.16.0")
implementation("androidx.appcompat:appcompat:1.7.1")
implementation("com.google.android.material:material:1.12.0")
implementation("androidx.constraintlayout:constraintlayout:2.2.1")
implementation("androidx.recyclerview:recyclerview:1.4.0")
implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.9.1")
implementation("androidx.browser:browser:1.8.0")
implementation("androidx.security:security-crypto:1.1.0-alpha06")
implementation("androidx.work:work-runtime-ktx:2.10.1")
implementation("androidx.room:room-runtime:2.7.1")
implementation("androidx.room:room-ktx:2.7.1")
ksp("androidx.room:room-compiler:2.7.1")
implementation("com.squareup.okhttp3:okhttp:4.12.0")
implementation("com.journeyapps:zxing-android-embedded:4.3.0")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2")
}

4
android/app/proguard-rules.pro vendored Normal file
View File

@@ -0,0 +1,4 @@
# zxing-android-embedded reflects into CaptureActivity subclasses; keep the
# whole client package rather than chasing individual members.
-keep class com.journeyapps.barcodescanner.** { *; }
-keep class com.google.zxing.** { *; }

View File

@@ -0,0 +1,55 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_DATA_SYNC" />
<uses-permission android:name="android.permission.CAMERA" />
<application
android:name=".App"
android:allowBackup="false"
android:icon="@mipmap/ic_launcher"
android:roundIcon="@mipmap/ic_launcher_round"
android:label="@string/app_name"
android:supportsRtl="true"
android:theme="@style/Theme.InternalUploader">
<activity
android:name=".MainActivity"
android:exported="true"
android:launchMode="singleTask">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<!-- Share-sheet target: anything shareable can be queued. -->
<intent-filter>
<action android:name="android.intent.action.SEND" />
<category android:name="android.intent.category.DEFAULT" />
<data android:mimeType="*/*" />
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.SEND_MULTIPLE" />
<category android:name="android.intent.category.DEFAULT" />
<data android:mimeType="*/*" />
</intent-filter>
<!-- In-app SSO pairing lands back here: internaluploader://pair?c=… -->
<intent-filter>
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="internaluploader" android:host="pair" />
</intent-filter>
</activity>
<!-- WorkManager's foreground service must declare the same type the
upload worker requests, or startForeground throws on 34+. -->
<service
android:name="androidx.work.impl.foreground.SystemForegroundService"
android:foregroundServiceType="dataSync"
tools:node="merge" />
</application>
</manifest>

View File

@@ -0,0 +1,24 @@
package eu.itaz.internaluploader
import android.app.Application
import android.app.NotificationChannel
import android.app.NotificationManager
class App : Application() {
override fun onCreate() {
super.onCreate()
val manager = getSystemService(NotificationManager::class.java)
manager.createNotificationChannel(
NotificationChannel(
CHANNEL_UPLOADS,
getString(R.string.channel_uploads),
NotificationManager.IMPORTANCE_LOW,
),
)
}
companion object {
const val CHANNEL_UPLOADS = "uploads"
}
}

View File

@@ -0,0 +1,214 @@
package eu.itaz.internaluploader
import android.Manifest
import android.content.Intent
import android.net.Uri
import android.os.Build
import android.os.Bundle
import android.view.Menu
import android.view.MenuItem
import android.view.View
import android.widget.Toast
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AppCompatActivity
import androidx.browser.customtabs.CustomTabsIntent
import androidx.core.net.toUri
import androidx.lifecycle.lifecycleScope
import androidx.recyclerview.widget.LinearLayoutManager
import com.journeyapps.barcodescanner.ScanContract
import com.journeyapps.barcodescanner.ScanOptions
import eu.itaz.internaluploader.data.AppDb
import eu.itaz.internaluploader.data.Status
import eu.itaz.internaluploader.databinding.ActivityMainBinding
import eu.itaz.internaluploader.pairing.PairApi
import eu.itaz.internaluploader.pairing.PairPayload
import eu.itaz.internaluploader.pairing.PairingStore
import eu.itaz.internaluploader.ui.QueueAdapter
import eu.itaz.internaluploader.upload.Enqueuer
import kotlinx.coroutines.flow.collectLatest
import kotlinx.coroutines.launch
class MainActivity : AppCompatActivity() {
private lateinit var binding: ActivityMainBinding
private lateinit var store: PairingStore
private val adapter = QueueAdapter()
private val scanQr = registerForActivityResult(ScanContract()) { result ->
val payload = result.contents?.let(PairPayload.Companion::fromQr)
if (payload == null) {
if (result.contents != null) toast(R.string.pair_qr_invalid)
} else {
completePairing(payload)
}
}
private val pickFiles =
registerForActivityResult(ActivityResultContracts.OpenMultipleDocuments()) { uris ->
uris.forEach(::enqueueUri)
}
private val askNotifications =
registerForActivityResult(ActivityResultContracts.RequestPermission()) { }
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = ActivityMainBinding.inflate(layoutInflater)
setContentView(binding.root)
setSupportActionBar(binding.toolbar)
store = PairingStore(this)
binding.queue.layoutManager = LinearLayoutManager(this)
binding.queue.adapter = adapter
binding.scanButton.setOnClickListener { startScan() }
binding.ssoButton.setOnClickListener { startAppSso() }
binding.addButton.setOnClickListener { pickFiles.launch(arrayOf("*/*")) }
binding.retryButton.setOnClickListener {
lifecycleScope.launch { Enqueuer.retryFailed(this@MainActivity) }
}
lifecycleScope.launch {
AppDb.get(this@MainActivity).uploads().observeAll().collectLatest { items ->
adapter.submitList(items)
binding.empty.visibility =
if (items.isEmpty() && store.current() != null) View.VISIBLE else View.GONE
binding.retryButton.visibility =
if (items.any { it.status == Status.FAILED }) View.VISIBLE else View.GONE
}
}
if (Build.VERSION.SDK_INT >= 33) {
askNotifications.launch(Manifest.permission.POST_NOTIFICATIONS)
}
render()
handleIntent(intent)
}
override fun onNewIntent(intent: Intent) {
super.onNewIntent(intent)
handleIntent(intent)
}
private fun handleIntent(intent: Intent) {
when (intent.action) {
Intent.ACTION_VIEW -> {
val code = intent.data?.getQueryParameter("c")
if (code != null) completePairing(PairPayload.fromDeepLinkCode(code))
}
Intent.ACTION_SEND -> {
sharedUri(intent)?.let(::enqueueUri)
}
Intent.ACTION_SEND_MULTIPLE -> {
sharedUris(intent).forEach(::enqueueUri)
}
}
// A share intent is consumed once; re-delivery on rotation would
// double-queue (dedupe by hash makes that harmless server-side, but
// the queue UI would still show it twice).
intent.action = null
}
@Suppress("DEPRECATION")
private fun sharedUri(intent: Intent): Uri? =
if (Build.VERSION.SDK_INT >= 33) {
intent.getParcelableExtra(Intent.EXTRA_STREAM, Uri::class.java)
} else {
intent.getParcelableExtra(Intent.EXTRA_STREAM)
}
@Suppress("DEPRECATION")
private fun sharedUris(intent: Intent): List<Uri> =
if (Build.VERSION.SDK_INT >= 33) {
intent.getParcelableArrayListExtra(Intent.EXTRA_STREAM, Uri::class.java) ?: emptyList()
} else {
intent.getParcelableArrayListExtra(Intent.EXTRA_STREAM) ?: emptyList()
}
private fun enqueueUri(uri: Uri) {
if (store.current() == null) {
toast(R.string.pair_first)
return
}
lifecycleScope.launch {
try {
Enqueuer.enqueue(this@MainActivity, uri)
} catch (e: Exception) {
Toast.makeText(
this@MainActivity,
getString(R.string.enqueue_failed, e.message ?: ""),
Toast.LENGTH_LONG,
).show()
}
}
}
private fun startScan() {
scanQr.launch(
ScanOptions()
.setDesiredBarcodeFormats(ScanOptions.QR_CODE)
.setPrompt(getString(R.string.pair_scan_prompt))
.setBeepEnabled(false)
.setOrientationLocked(false),
)
}
private fun startAppSso() {
CustomTabsIntent.Builder().build()
.launchUrl(this, "${BuildConfig.DEFAULT_SERVER_URL}/pair/app".toUri())
}
private fun completePairing(payload: PairPayload) {
lifecycleScope.launch {
try {
store.save(PairApi.pair(payload))
toast(R.string.pair_success)
} catch (e: Exception) {
Toast.makeText(
this@MainActivity,
getString(R.string.pair_failed, e.message ?: ""),
Toast.LENGTH_LONG,
).show()
}
render()
}
}
private fun render() {
val paired = store.current()
binding.pairGroup.visibility = if (paired == null) View.VISIBLE else View.GONE
binding.uploadGroup.visibility = if (paired == null) View.GONE else View.VISIBLE
binding.pairedAs.text =
if (paired == null) "" else getString(R.string.paired_as, paired.name)
invalidateOptionsMenu()
}
override fun onCreateOptionsMenu(menu: Menu): Boolean {
menuInflater.inflate(R.menu.main_menu, menu)
val paired = store.current() != null
menu.findItem(R.id.action_unpair).isVisible = paired
menu.findItem(R.id.action_clear_finished).isVisible = paired
return true
}
override fun onOptionsItemSelected(item: MenuItem): Boolean =
when (item.itemId) {
R.id.action_unpair -> {
store.clear()
render()
true
}
R.id.action_clear_finished -> {
lifecycleScope.launch {
AppDb.get(this@MainActivity).uploads().deleteByStatus(Status.DONE)
}
true
}
else -> super.onOptionsItemSelected(item)
}
private fun toast(resId: Int) {
Toast.makeText(this, resId, Toast.LENGTH_SHORT).show()
}
}

View File

@@ -0,0 +1,75 @@
package eu.itaz.internaluploader.data
import android.content.Context
import androidx.room.Dao
import androidx.room.Database
import androidx.room.Entity
import androidx.room.Insert
import androidx.room.PrimaryKey
import androidx.room.Query
import androidx.room.Room
import androidx.room.RoomDatabase
import kotlinx.coroutines.flow.Flow
object Status {
const val PENDING = "pending"
const val UPLOADING = "uploading"
const val DONE = "done"
const val FAILED = "failed"
}
@Entity(tableName = "uploads")
data class UploadItem(
@PrimaryKey(autoGenerate = true) val id: Long = 0,
/** What the user shared — display only; never sent to the server. */
val originalName: String,
/** `<sha256>.<ext>` — the only name the server ever sees. */
val remoteName: String,
/** Private copy the worker uploads from; survives source URI revocation. */
val cachePath: String,
val size: Long,
val status: String = Status.PENDING,
val progress: Int = 0,
val error: String? = null,
val createdAt: Long = System.currentTimeMillis(),
)
@Dao
interface UploadDao {
@Insert
suspend fun insert(item: UploadItem): Long
@Query("SELECT * FROM uploads ORDER BY createdAt DESC")
fun observeAll(): Flow<List<UploadItem>>
@Query("SELECT * FROM uploads WHERE id = :id")
suspend fun byId(id: Long): UploadItem?
@Query("SELECT * FROM uploads WHERE status = :status")
suspend fun byStatus(status: String): List<UploadItem>
@Query("UPDATE uploads SET status = :status, progress = :progress, error = :error WHERE id = :id")
suspend fun setStatus(id: Long, status: String, progress: Int = 0, error: String? = null)
@Query("DELETE FROM uploads WHERE status = :status")
suspend fun deleteByStatus(status: String)
}
@Database(entities = [UploadItem::class], version = 1, exportSchema = false)
abstract class AppDb : RoomDatabase() {
abstract fun uploads(): UploadDao
companion object {
@Volatile
private var instance: AppDb? = null
fun get(context: Context): AppDb =
instance ?: synchronized(this) {
instance ?: Room.databaseBuilder(
context.applicationContext,
AppDb::class.java,
"internaluploader.db",
).build().also { instance = it }
}
}
}

View File

@@ -0,0 +1,119 @@
package eu.itaz.internaluploader.pairing
import android.content.Context
import android.content.SharedPreferences
import androidx.security.crypto.EncryptedSharedPreferences
import androidx.security.crypto.MasterKey
import eu.itaz.internaluploader.BuildConfig
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import org.json.JSONObject
data class PairedState(val serverUrl: String, val token: String, val name: String)
/**
* Device credential store. The server URL lives here too — it arrives inside
* the pairing QR (or is the baked-in default for the in-app SSO path) and is
* deliberately never surfaced anywhere in the UI.
*/
class PairingStore(context: Context) {
private val prefs: SharedPreferences
init {
val masterKey = MasterKey.Builder(context)
.setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
.build()
prefs = EncryptedSharedPreferences.create(
context,
"pairing",
masterKey,
EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM,
)
}
fun current(): PairedState? {
val url = prefs.getString(KEY_URL, null) ?: return null
val token = prefs.getString(KEY_TOKEN, null) ?: return null
val name = prefs.getString(KEY_NAME, null) ?: return null
return PairedState(url, token, name)
}
fun save(state: PairedState) {
prefs.edit()
.putString(KEY_URL, state.serverUrl)
.putString(KEY_TOKEN, state.token)
.putString(KEY_NAME, state.name)
.apply()
}
fun clear() {
prefs.edit().clear().apply()
}
private companion object {
const val KEY_URL = "serverUrl"
const val KEY_TOKEN = "token"
const val KEY_NAME = "name"
}
}
/** QR / deep-link payload: {"v":1,"u":"https://…","c":"<one-time code>"}. */
data class PairPayload(val serverUrl: String, val code: String) {
companion object {
fun fromQr(text: String): PairPayload? = try {
val json = JSONObject(text)
if (json.optInt("v") != 1) null
else {
val url = json.optString("u")
val code = json.optString("c")
if (url.startsWith("https://") && code.isNotEmpty()) {
PairPayload(url.trimEnd('/'), code)
} else null
}
} catch (_: Exception) {
null
}
/** Deep link carries only the code; the origin is the baked-in default. */
fun fromDeepLinkCode(code: String): PairPayload =
PairPayload(BuildConfig.DEFAULT_SERVER_URL, code)
}
}
class PairingException(message: String) : Exception(message)
object PairApi {
private val client = OkHttpClient()
private val jsonType = "application/json".toMediaType()
/** Exchange a one-time pairing code for the long-lived device token. */
suspend fun pair(payload: PairPayload): PairedState = withContext(Dispatchers.IO) {
val body = JSONObject().put("code", payload.code).toString().toRequestBody(jsonType)
val request = Request.Builder()
.url("${payload.serverUrl}/api/pair")
.post(body)
.build()
client.newCall(request).execute().use { response ->
when {
response.isSuccessful -> {
val json = JSONObject(response.body!!.string())
PairedState(
serverUrl = payload.serverUrl,
token = json.getString("token"),
name = json.getString("name"),
)
}
response.code == 410 || response.code == 400 ->
throw PairingException("code expired or already used")
else -> throw PairingException("pairing failed (HTTP ${response.code})")
}
}
}
}

View File

@@ -0,0 +1,51 @@
package eu.itaz.internaluploader.ui
import android.text.format.Formatter
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import androidx.recyclerview.widget.DiffUtil
import androidx.recyclerview.widget.ListAdapter
import androidx.recyclerview.widget.RecyclerView
import eu.itaz.internaluploader.R
import eu.itaz.internaluploader.data.Status
import eu.itaz.internaluploader.data.UploadItem
import eu.itaz.internaluploader.databinding.ItemUploadBinding
class QueueAdapter : ListAdapter<UploadItem, QueueAdapter.Holder>(Diff) {
class Holder(val binding: ItemUploadBinding) : RecyclerView.ViewHolder(binding.root)
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): Holder =
Holder(ItemUploadBinding.inflate(LayoutInflater.from(parent.context), parent, false))
override fun onBindViewHolder(holder: Holder, position: Int) {
val item = getItem(position)
val context = holder.binding.root.context
holder.binding.name.text = item.originalName
holder.binding.size.text = Formatter.formatShortFileSize(context, item.size)
holder.binding.progress.visibility =
if (item.status == Status.UPLOADING) View.VISIBLE else View.GONE
holder.binding.progress.isIndeterminate = item.progress == 0
holder.binding.progress.progress = item.progress
holder.binding.status.text = when (item.status) {
Status.PENDING -> context.getString(R.string.status_pending)
Status.UPLOADING -> context.getString(R.string.status_uploading, item.progress)
Status.DONE -> context.getString(R.string.status_done)
Status.FAILED -> context.getString(
R.string.status_failed,
item.error ?: context.getString(R.string.status_failed_unknown),
)
else -> item.status
}
holder.binding.check.visibility =
if (item.status == Status.DONE) View.VISIBLE else View.GONE
}
private object Diff : DiffUtil.ItemCallback<UploadItem>() {
override fun areItemsTheSame(a: UploadItem, b: UploadItem) = a.id == b.id
override fun areContentsTheSame(a: UploadItem, b: UploadItem) = a == b
}
}

View File

@@ -0,0 +1,45 @@
package eu.itaz.internaluploader.upload
import okhttp3.MediaType
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.RequestBody
import okio.BufferedSink
import okio.source
import java.io.File
/**
* Streams a file as the request body, reporting bytes-sent roughly once per
* second so progress writes don't outnumber the payload reads.
*/
class CountingFileRequestBody(
private val file: File,
private val onProgress: (bytesSent: Long) -> Unit,
) : RequestBody() {
override fun contentType(): MediaType = "application/octet-stream".toMediaType()
override fun contentLength(): Long = file.length()
override fun writeTo(sink: BufferedSink) {
var sent = 0L
var lastReport = 0L
file.source().use { source ->
while (true) {
val read = source.read(sink.buffer, SEGMENT)
if (read < 0) break
sent += read
sink.emitCompleteSegments()
val now = System.currentTimeMillis()
if (now - lastReport >= 1000) {
lastReport = now
onProgress(sent)
}
}
}
onProgress(sent)
}
private companion object {
const val SEGMENT = 64L * 1024
}
}

View File

@@ -0,0 +1,111 @@
package eu.itaz.internaluploader.upload
import android.content.Context
import android.net.Uri
import android.provider.OpenableColumns
import androidx.work.BackoffPolicy
import androidx.work.Constraints
import androidx.work.ExistingWorkPolicy
import androidx.work.NetworkType
import androidx.work.OneTimeWorkRequestBuilder
import androidx.work.WorkManager
import androidx.work.workDataOf
import eu.itaz.internaluploader.data.AppDb
import eu.itaz.internaluploader.data.Status
import eu.itaz.internaluploader.data.UploadItem
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import java.io.File
import java.security.MessageDigest
import java.util.UUID
import java.util.concurrent.TimeUnit
object Enqueuer {
/**
* Copy the shared content into app-private storage while hashing it, so a
* retry never depends on the source URI still being readable, then hand
* the file to WorkManager.
*/
suspend fun enqueue(context: Context, uri: Uri): UploadItem = withContext(Dispatchers.IO) {
val originalName = displayNameOf(context, uri) ?: "unnamed"
val queueDir = File(context.filesDir, "queue").apply { mkdirs() }
val cacheFile = File(queueDir, UUID.randomUUID().toString())
val digest = MessageDigest.getInstance("SHA-256")
var size = 0L
context.contentResolver.openInputStream(uri).use { input ->
requireNotNull(input) { "source stream is gone" }
cacheFile.outputStream().use { output ->
val buffer = ByteArray(1 shl 16)
while (true) {
val read = input.read(buffer)
if (read < 0) break
digest.update(buffer, 0, read)
output.write(buffer, 0, read)
size += read
}
}
}
val hash = digest.digest().joinToString("") { "%02x".format(it) }
val ext = originalName.substringAfterLast('.', "").lowercase()
.takeIf { it.isNotEmpty() && it.length <= 16 && it.all { c -> c.isLetterOrDigit() } }
val remoteName = if (ext != null) "$hash.$ext" else hash
val db = AppDb.get(context)
val item = UploadItem(
originalName = originalName,
remoteName = remoteName,
cachePath = cacheFile.absolutePath,
size = size,
)
val id = db.uploads().insert(item)
schedule(context, id)
item.copy(id = id)
}
/** (Re-)schedule the worker for one queue item. */
fun schedule(context: Context, itemId: Long) {
val request = OneTimeWorkRequestBuilder<UploadWorker>()
.setInputData(workDataOf(UploadWorker.KEY_ITEM_ID to itemId))
.setConstraints(
Constraints.Builder().setRequiredNetworkType(NetworkType.CONNECTED).build(),
)
.setBackoffCriteria(BackoffPolicy.EXPONENTIAL, 30, TimeUnit.SECONDS)
.build()
WorkManager.getInstance(context)
.enqueueUniqueWork("upload-$itemId", ExistingWorkPolicy.KEEP, request)
}
suspend fun retryFailed(context: Context) {
val db = AppDb.get(context)
for (item in db.uploads().byStatus(Status.FAILED)) {
if (File(item.cachePath).exists()) {
db.uploads().setStatus(item.id, Status.PENDING)
// REPLACE: the previous unique work chain for this id has already
// terminated in failure; KEEP would ignore the new request.
val request = OneTimeWorkRequestBuilder<UploadWorker>()
.setInputData(workDataOf(UploadWorker.KEY_ITEM_ID to item.id))
.setConstraints(
Constraints.Builder().setRequiredNetworkType(NetworkType.CONNECTED).build(),
)
.setBackoffCriteria(BackoffPolicy.EXPONENTIAL, 30, TimeUnit.SECONDS)
.build()
WorkManager.getInstance(context)
.enqueueUniqueWork("upload-${item.id}", ExistingWorkPolicy.REPLACE, request)
}
}
}
private fun displayNameOf(context: Context, uri: Uri): String? {
context.contentResolver.query(uri, null, null, null, null)?.use { cursor ->
val index = cursor.getColumnIndex(OpenableColumns.DISPLAY_NAME)
if (index >= 0 && cursor.moveToFirst()) {
return cursor.getString(index)
}
}
return uri.lastPathSegment
}
}

View File

@@ -0,0 +1,164 @@
package eu.itaz.internaluploader.upload
import android.app.Notification
import android.content.Context
import android.content.pm.ServiceInfo
import android.os.Build
import androidx.core.app.NotificationCompat
import androidx.work.CoroutineWorker
import androidx.work.ForegroundInfo
import androidx.work.WorkerParameters
import androidx.work.workDataOf
import eu.itaz.internaluploader.App
import eu.itaz.internaluploader.R
import eu.itaz.internaluploader.data.AppDb
import eu.itaz.internaluploader.data.Status
import eu.itaz.internaluploader.data.UploadItem
import eu.itaz.internaluploader.pairing.PairingStore
import okhttp3.OkHttpClient
import okhttp3.Request
import java.io.File
import java.io.IOException
import java.util.concurrent.TimeUnit
class UploadWorker(context: Context, params: WorkerParameters) :
CoroutineWorker(context, params) {
private val db = AppDb.get(context)
override suspend fun doWork(): Result {
val itemId = inputData.getLong(KEY_ITEM_ID, -1)
val item = db.uploads().byId(itemId) ?: return Result.failure()
if (item.status == Status.DONE) return Result.success()
val paired = PairingStore(applicationContext).current()
?: return permanentFailure(item, applicationContext.getString(R.string.error_not_paired))
val file = File(item.cachePath)
if (!file.exists()) {
return permanentFailure(item, applicationContext.getString(R.string.error_cache_gone))
}
// Long uploads outlive the default 10-minute background budget; run as
// a dataSync foreground job with visible progress. Starting foreground
// can be refused (e.g. app freshly in background on 12+) — the upload
// itself still proceeds, just without a notification.
runCatching { setForeground(foregroundInfo(item, 0)) }
db.uploads().setStatus(item.id, Status.UPLOADING)
return try {
when (val outcome = upload(paired.serverUrl, paired.token, item, file)) {
UploadOutcome.DONE -> {
db.uploads().setStatus(item.id, Status.DONE, progress = 100)
file.delete()
Result.success()
}
UploadOutcome.UNAUTHORIZED ->
permanentFailure(item, applicationContext.getString(R.string.error_unauthorized))
UploadOutcome.REJECTED ->
permanentFailure(item, applicationContext.getString(R.string.error_rejected))
is UploadOutcome.TRANSIENT -> {
db.uploads().setStatus(item.id, Status.PENDING, error = outcome.message)
Result.retry()
}
}
} catch (e: IOException) {
db.uploads().setStatus(item.id, Status.PENDING, error = e.message)
Result.retry()
}
}
private sealed interface UploadOutcome {
data object DONE : UploadOutcome
data object UNAUTHORIZED : UploadOutcome
data object REJECTED : UploadOutcome
data class TRANSIENT(val message: String) : UploadOutcome
}
private suspend fun upload(
serverUrl: String,
token: String,
item: UploadItem,
file: File,
): UploadOutcome {
// A retried upload may already have landed: ask before sending bytes.
val head = Request.Builder()
.url("$serverUrl/api/object/${item.remoteName}")
.head()
.header("Authorization", "Bearer $token")
.build()
client.newCall(head).execute().use { response ->
when (response.code) {
204 -> return UploadOutcome.DONE
401 -> return UploadOutcome.UNAUTHORIZED
404 -> Unit // expected: not there yet
else -> return UploadOutcome.TRANSIENT("HEAD ${response.code}")
}
}
val body = CountingFileRequestBody(file) { sent ->
val percent = if (item.size > 0) ((sent * 100) / item.size).toInt() else 0
setProgressAsync(workDataOf(KEY_PROGRESS to percent))
runCatching {
setForegroundAsync(foregroundInfo(item, percent))
}
// Room write per callback would thrash; CountingFileRequestBody
// already throttles callbacks to ~1/s.
kotlinx.coroutines.runBlocking {
db.uploads().setStatus(item.id, Status.UPLOADING, progress = percent)
}
}
val put = Request.Builder()
.url("$serverUrl/api/upload?name=${item.remoteName}")
.put(body)
.header("Authorization", "Bearer $token")
.build()
client.newCall(put).execute().use { response ->
return when (response.code) {
200, 201 -> UploadOutcome.DONE
401 -> UploadOutcome.UNAUTHORIZED
409, 413, 400 -> UploadOutcome.REJECTED
else -> UploadOutcome.TRANSIENT("PUT ${response.code}")
}
}
}
private suspend fun permanentFailure(item: UploadItem, message: String): Result {
db.uploads().setStatus(item.id, Status.FAILED, error = message)
return Result.failure()
}
private fun foregroundInfo(item: UploadItem, percent: Int): ForegroundInfo {
val notification: Notification =
NotificationCompat.Builder(applicationContext, App.CHANNEL_UPLOADS)
.setSmallIcon(R.drawable.ic_notification)
.setContentTitle(applicationContext.getString(R.string.notification_uploading))
.setContentText(item.originalName)
.setProgress(100, percent, percent == 0)
.setOngoing(true)
.setOnlyAlertOnce(true)
.build()
return if (Build.VERSION.SDK_INT >= 29) {
ForegroundInfo(
item.id.toInt(),
notification,
ServiceInfo.FOREGROUND_SERVICE_TYPE_DATA_SYNC,
)
} else {
ForegroundInfo(item.id.toInt(), notification)
}
}
companion object {
const val KEY_ITEM_ID = "itemId"
const val KEY_PROGRESS = "progress"
private val client = OkHttpClient.Builder()
.connectTimeout(30, TimeUnit.SECONDS)
.readTimeout(2, TimeUnit.MINUTES)
// A multi-GB PUT writes for as long as it writes.
.writeTimeout(0, TimeUnit.MILLISECONDS)
.build()
}
}

View File

@@ -0,0 +1,9 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="24dp"
android:height="24dp"
android:viewportWidth="24"
android:viewportHeight="24">
<path
android:fillColor="#FF000000"
android:pathData="M9,16.17L4.83,12l-1.42,1.41L9,19 21,7l-1.41,-1.41z" />
</vector>

View File

@@ -0,0 +1,15 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="108dp"
android:height="108dp"
android:viewportWidth="108"
android:viewportHeight="108">
<!-- Up arrow into a tray: one-way upload. -->
<group android:translateX="30" android:translateY="28">
<path
android:fillColor="#FFFFFF"
android:pathData="M24,0 L40,18 L31,18 L31,34 L17,34 L17,18 L8,18 Z" />
<path
android:fillColor="#FFFFFF"
android:pathData="M0,40 L8,40 L8,44 L40,44 L40,40 L48,40 L48,52 L0,52 Z" />
</group>
</vector>

View File

@@ -0,0 +1,9 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="24dp"
android:height="24dp"
android:viewportWidth="24"
android:viewportHeight="24">
<path
android:fillColor="#FFFFFFFF"
android:pathData="M9,16h6v-6h4l-7,-7 -7,7h4zM5,18h14v2L5,20z" />
</vector>

View File

@@ -0,0 +1,122 @@
<?xml version="1.0" encoding="utf-8"?>
<androidx.coordinatorlayout.widget.CoordinatorLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:layout_width="match_parent"
android:layout_height="match_parent">
<com.google.android.material.appbar.AppBarLayout
android:layout_width="match_parent"
android:layout_height="wrap_content">
<com.google.android.material.appbar.MaterialToolbar
android:id="@+id/toolbar"
android:layout_width="match_parent"
android:layout_height="?attr/actionBarSize"
app:title="@string/app_name" />
</com.google.android.material.appbar.AppBarLayout>
<androidx.constraintlayout.widget.ConstraintLayout
android:layout_width="match_parent"
android:layout_height="match_parent"
app:layout_behavior="@string/appbar_scrolling_view_behavior">
<!-- Unpaired: the two ways in. -->
<LinearLayout
android:id="@+id/pairGroup"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:orientation="vertical"
android:gravity="center_horizontal"
android:padding="32dp"
app:layout_constraintTop_toTopOf="parent"
app:layout_constraintBottom_toBottomOf="parent">
<ImageView
android:layout_width="96dp"
android:layout_height="96dp"
android:importantForAccessibility="no"
android:src="@drawable/ic_launcher_foreground" />
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginTop="16dp"
android:text="@string/pair_explainer"
android:textAlignment="center"
android:textAppearance="?attr/textAppearanceBodyLarge" />
<com.google.android.material.button.MaterialButton
android:id="@+id/scanButton"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginTop="32dp"
android:text="@string/pair_scan_button" />
<com.google.android.material.button.MaterialButton
android:id="@+id/ssoButton"
style="@style/Widget.Material3.Button.OutlinedButton"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginTop="12dp"
android:text="@string/pair_sso_button" />
</LinearLayout>
<!-- Paired: the queue. -->
<LinearLayout
android:id="@+id/uploadGroup"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:orientation="vertical"
android:visibility="gone">
<TextView
android:id="@+id/pairedAs"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:paddingHorizontal="16dp"
android:paddingVertical="8dp"
android:textAppearance="?attr/textAppearanceBodyMedium" />
<LinearLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:orientation="horizontal"
android:paddingHorizontal="16dp">
<com.google.android.material.button.MaterialButton
android:id="@+id/addButton"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:layout_weight="1"
android:text="@string/add_files" />
<com.google.android.material.button.MaterialButton
android:id="@+id/retryButton"
style="@style/Widget.Material3.Button.OutlinedButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginStart="8dp"
android:text="@string/retry_failed"
android:visibility="gone" />
</LinearLayout>
<TextView
android:id="@+id/empty"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:padding="32dp"
android:text="@string/queue_empty"
android:textAlignment="center"
android:visibility="gone" />
<androidx.recyclerview.widget.RecyclerView
android:id="@+id/queue"
android:layout_width="match_parent"
android:layout_height="0dp"
android:layout_weight="1"
android:clipToPadding="false"
android:paddingBottom="16dp" />
</LinearLayout>
</androidx.constraintlayout.widget.ConstraintLayout>
</androidx.coordinatorlayout.widget.CoordinatorLayout>

View File

@@ -0,0 +1,60 @@
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:paddingHorizontal="16dp"
android:paddingVertical="10dp">
<TextView
android:id="@+id/name"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:ellipsize="middle"
android:singleLine="true"
android:textAppearance="?attr/textAppearanceBodyLarge"
app:layout_constraintEnd_toStartOf="@id/check"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
<ImageView
android:id="@+id/check"
android:layout_width="24dp"
android:layout_height="24dp"
android:importantForAccessibility="no"
android:src="@drawable/ic_check"
android:visibility="gone"
app:layout_constraintBottom_toBottomOf="@id/name"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintTop_toTopOf="@id/name"
app:tint="?attr/colorPrimary" />
<TextView
android:id="@+id/status"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:textAppearance="?attr/textAppearanceBodySmall"
app:layout_constraintEnd_toStartOf="@id/size"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@id/name" />
<TextView
android:id="@+id/size"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:textAppearance="?attr/textAppearanceBodySmall"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintTop_toBottomOf="@id/name" />
<ProgressBar
android:id="@+id/progress"
style="?android:attr/progressBarStyleHorizontal"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:layout_marginTop="4dp"
android:visibility="gone"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@id/status" />
</androidx.constraintlayout.widget.ConstraintLayout>

View File

@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto">
<item
android:id="@+id/action_clear_finished"
android:title="@string/action_clear_finished"
app:showAsAction="never" />
<item
android:id="@+id/action_unpair"
android:title="@string/action_unpair"
app:showAsAction="never" />
</menu>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@color/ic_launcher_background" />
<foreground android:drawable="@drawable/ic_launcher_foreground" />
</adaptive-icon>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@color/ic_launcher_background" />
<foreground android:drawable="@drawable/ic_launcher_foreground" />
</adaptive-icon>

View File

@@ -0,0 +1,5 @@
<resources>
<color name="brand">#2E5AAC</color>
<color name="white">#FFFFFFFF</color>
<color name="ic_launcher_background">#2E5AAC</color>
</resources>

View File

@@ -0,0 +1,35 @@
<resources>
<string name="app_name">InternalUploader</string>
<string name="channel_uploads">Uploads</string>
<string name="pair_explainer">Pair this device to start uploading. Sign in on any browser and scan the code, or sign in right here.</string>
<string name="pair_scan_button">Scan QR code</string>
<string name="pair_sso_button">Sign in</string>
<string name="pair_scan_prompt">Scan the pairing code from your browser</string>
<string name="pair_qr_invalid">Not a pairing code</string>
<string name="pair_success">Paired</string>
<string name="pair_failed">Pairing failed: %1$s</string>
<string name="pair_first">Pair this device first</string>
<string name="paired_as">Paired as %1$s</string>
<string name="add_files">Add files</string>
<string name="retry_failed">Retry failed</string>
<string name="queue_empty">Nothing queued. Share files to InternalUploader or add them here.</string>
<string name="enqueue_failed">Could not queue file: %1$s</string>
<string name="status_pending">Waiting…</string>
<string name="status_uploading">Uploading… %1$d%%</string>
<string name="status_done">Uploaded</string>
<string name="status_failed">Failed: %1$s</string>
<string name="status_failed_unknown">unknown error</string>
<string name="notification_uploading">Uploading</string>
<string name="error_not_paired">device is not paired</string>
<string name="error_cache_gone">queued copy is gone</string>
<string name="error_unauthorized">rejected — unpair and pair again</string>
<string name="error_rejected">rejected by server</string>
<string name="action_unpair">Unpair</string>
<string name="action_clear_finished">Clear finished</string>
</resources>

View File

@@ -0,0 +1,6 @@
<resources>
<style name="Theme.InternalUploader" parent="Theme.Material3.DayNight.NoActionBar">
<item name="colorPrimary">@color/brand</item>
<item name="colorOnPrimary">@color/white</item>
</style>
</resources>

5
android/build.gradle.kts Normal file
View File

@@ -0,0 +1,5 @@
plugins {
id("com.android.application") version "8.11.1" apply false
id("org.jetbrains.kotlin.android") version "2.1.21" apply false
id("com.google.devtools.ksp") version "2.1.21-2.0.1" apply false
}

View File

@@ -0,0 +1,4 @@
org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8
android.useAndroidX=true
android.nonTransitiveRClass=true
kotlin.code.style=official

Binary file not shown.

View File

@@ -0,0 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.14.3-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

251
android/gradlew vendored Executable file
View File

@@ -0,0 +1,251 @@
#!/bin/sh
#
# Copyright © 2015-2021 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
#
##############################################################################
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
CLASSPATH="\\\"\\\""
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
if ! command -v java >/dev/null 2>&1
then
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Collect all arguments for the java command:
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
# and any embedded shellness will be escaped.
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
# treated as '${Hostname}' itself on the command line.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-classpath "$CLASSPATH" \
-jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"

94
android/gradlew.bat vendored Normal file
View File

@@ -0,0 +1,94 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@rem SPDX-License-Identifier: Apache-2.0
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:execute
@rem Setup the command line
set CLASSPATH=
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %*
:end
@rem End local scope for the variables with windows NT shell
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

View File

@@ -0,0 +1,18 @@
pluginManagement {
repositories {
google()
mavenCentral()
gradlePluginPortal()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
}
}
rootProject.name = "internaluploader"
include(":app")

37
server/Dockerfile Normal file
View File

@@ -0,0 +1,37 @@
# internaluploader-server. Build context is server/.
#
# Registries are fully qualified: podman enforces short-name resolution and
# fails a non-interactive build with "cannot prompt without a TTY" rather than
# choosing one.
FROM docker.io/library/node:22-alpine AS build
WORKDIR /src
COPY package.json package-lock.json ./
RUN npm ci
COPY tsconfig.json tsconfig.test.json ./
COPY src ./src
COPY test ./test
# Typecheck and test INSIDE the image build — an image that compiles but fails
# its own tests should not reach a registry.
RUN npm run typecheck
RUN npm test
RUN npm run build
# Drop devDependencies from what gets copied forward.
RUN npm prune --omit=dev
FROM docker.io/library/node:22-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY --from=build /src/node_modules ./node_modules
COPY --from=build /src/dist ./dist
COPY --from=build /src/package.json ./package.json
# uid 1000 — the same uid/gid the metube container writes /downloads with, so
# both apps' files on the share look identical.
USER node
EXPOSE 8080
CMD ["node", "dist/index.js"]

14
server/build.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Build and push the server image. Tag = <date>-<git sha>, matching the
# convention of the other mysources.co.uk images (e.g. authentik-qr-login).
set -euo pipefail
cd "$(dirname "$0")"
REGISTRY="mysources.co.uk/michal"
NAME="internaluploader-server"
TAG="$(date +%Y%m%d)-$(git rev-parse --short=12 HEAD)"
IMAGE="${REGISTRY}/${NAME}:${TAG}"
podman build -t "${IMAGE}" .
podman push "${IMAGE}"
echo "pushed ${IMAGE}"

1569
server/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

29
server/package.json Normal file
View File

@@ -0,0 +1,29 @@
{
"name": "internaluploader-server",
"version": "1.0.0",
"private": true,
"description": "One-way upload endpoint for the InternalUploader Android app: SSO-paired devices drop files into the metube directory as <sha256>.<ext>. No listing, no download.",
"main": "dist/index.js",
"scripts": {
"build": "tsc",
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.test.json",
"test": "node --test --experimental-strip-types test/*.test.ts",
"start": "node dist/index.js"
},
"engines": {
"node": "22.x"
},
"dependencies": {
"cookie-session": "^2.1.0",
"express": "^5.1.0",
"openid-client": "^5.6.5",
"qrcode": "^1.5.4"
},
"devDependencies": {
"@types/cookie-session": "^2.0.49",
"@types/express": "^5.0.6",
"@types/node": "^22.10.0",
"@types/qrcode": "^1.5.5",
"typescript": "^5.9.3"
}
}

50
server/src/config.ts Normal file
View File

@@ -0,0 +1,50 @@
/** All configuration is environment variables; the deployment cites each name. */
export interface Config {
port: number;
/** e.g. https://puploader.ad.itaz.eu — used for the OIDC redirect and QR payload. */
publicOrigin: string;
oidc: {
issuer: string;
clientId: string;
clientSecret: string;
requiredGroup: string;
};
cookieSecret: string;
/** Where finished files land — the metube download directory bind mount. */
downloadsDir: string;
/** Persistent state (device token store). A docker named volume. */
stateDir: string;
maxUploadBytes: number;
/** Deep-link scheme the Android app registers, for /pair/app. */
appScheme: string;
pairingCodeTtlSeconds: number;
}
function required(name: string): string {
const value = process.env[name];
if (!value) {
throw new Error(`Missing required environment variable ${name}`);
}
return value;
}
export function loadConfig(): Config {
return {
port: Number(process.env.PU_PORT ?? 8080),
publicOrigin: required("PU_PUBLIC_ORIGIN").replace(/\/$/, ""),
oidc: {
issuer: required("PU_OIDC_ISSUER"),
clientId: required("PU_OIDC_CLIENT_ID"),
clientSecret: required("PU_OIDC_CLIENT_SECRET"),
requiredGroup: required("PU_REQUIRED_GROUP"),
},
cookieSecret: required("PU_COOKIE_SECRET"),
downloadsDir: process.env.PU_DOWNLOADS_DIR ?? "/downloads",
stateDir: process.env.PU_STATE_DIR ?? "/state",
// 32 GiB default: large enough for any single video, small enough that a
// runaway client cannot fill the volume in one request unnoticed.
maxUploadBytes: Number(process.env.PU_MAX_UPLOAD_BYTES ?? 32 * 1024 ** 3),
appScheme: process.env.PU_APP_SCHEME ?? "internaluploader",
pairingCodeTtlSeconds: Number(process.env.PU_PAIRING_CODE_TTL_SECONDS ?? 300),
};
}

178
server/src/index.ts Normal file
View File

@@ -0,0 +1,178 @@
import cookieSession from "cookie-session";
import express, { type NextFunction, type Request, type Response } from "express";
import { loadConfig } from "./config.ts";
import { beginAuth, completeAuth, isAuthorised } from "./oidc.ts";
import { deniedPage, pairingPage, qrPayload } from "./pages.ts";
import { TokenStore } from "./tokens.ts";
import { UploadDir, isValidObjectName } from "./upload.ts";
const config = loadConfig();
const tokens = new TokenStore(config.stateDir, config.pairingCodeTtlSeconds * 1000);
const uploads = new UploadDir(config.downloadsDir, config.maxUploadBytes);
const app = express();
app.disable("x-powered-by");
// TLS terminates at the Cilium ingress; without this `secure` cookies die.
app.set("trust proxy", 1);
app.use(
cookieSession({
name: "puploader",
secret: config.cookieSecret,
httpOnly: true,
sameSite: "lax",
secure: true,
maxAge: 8 * 60 * 60 * 1000,
}),
);
app.get("/healthz", (_req, res) => {
res.type("text/plain").send("ok");
});
/** Browser SSO gate: redirects to authentik when there is no session. */
function requireBrowserSso(req: Request, res: Response, next: NextFunction): void {
const session = req.session as CookieSessionInterfaces.CookieSessionObject;
if (typeof session.username === "string") {
next();
return;
}
beginAuth(config)
.then((auth) => {
session.oidcState = auth.state;
session.oidcVerifier = auth.codeVerifier;
session.returnTo = req.path;
res.redirect(auth.url);
})
.catch(next);
}
app.get("/oidc/callback", (req, res, next) => {
const session = req.session as CookieSessionInterfaces.CookieSessionObject;
const state = session.oidcState;
const codeVerifier = session.oidcVerifier;
if (typeof state !== "string" || typeof codeVerifier !== "string") {
res.status(400).type("text/plain").send("no login in progress");
return;
}
completeAuth(config, req, { state, codeVerifier })
.then((identity) => {
delete session.oidcState;
delete session.oidcVerifier;
if (!isAuthorised(config, identity)) {
res.status(403).type("html").send(deniedPage(identity.username, config.oidc.requiredGroup));
return;
}
session.username = identity.username;
const returnTo = typeof session.returnTo === "string" ? session.returnTo : "/";
delete session.returnTo;
res.redirect(returnTo);
})
.catch(next);
});
app.get("/", requireBrowserSso, (req, res, next) => {
const session = req.session as CookieSessionInterfaces.CookieSessionObject;
const username = session.username as string;
const pairing = tokens.issuePairingCode(username);
pairingPage(username, qrPayload(config.publicOrigin, pairing.code), config.pairingCodeTtlSeconds)
.then((html) => res.type("html").send(html))
.catch(next);
});
/**
* App-initiated SSO: the app opens this in a Custom Tab; after the same
* browser login the pairing code goes back through the deep link. The app
* never sees OIDC — the phone's browser does, exactly like the QR path.
*/
app.get("/pair/app", requireBrowserSso, (req, res) => {
const session = req.session as CookieSessionInterfaces.CookieSessionObject;
const pairing = tokens.issuePairingCode(session.username as string);
res.redirect(`${config.appScheme}://pair?c=${pairing.code}`);
});
app.post("/api/pair", express.json({ limit: "4kb" }), (req, res) => {
const code = (req.body as { code?: unknown })?.code;
if (typeof code !== "string" || !/^[0-9a-f]{64}$/.test(code)) {
res.status(400).json({ error: "malformed pairing code" });
return;
}
const redeemed = tokens.redeemPairingCode(code);
if (!redeemed) {
res.status(410).json({ error: "pairing code expired or already used" });
return;
}
console.log(`paired device #${tokens.deviceCount} for ${redeemed.username}`);
res.json({ token: redeemed.token, name: redeemed.username });
});
/** Bearer gate for the upload API. */
function requireDeviceToken(req: Request, res: Response, next: NextFunction): void {
const header = req.headers.authorization ?? "";
const token = header.startsWith("Bearer ") ? header.slice(7) : undefined;
const username = token ? tokens.authenticate(token) : undefined;
if (!username) {
res.status(401).json({ error: "invalid device token" });
return;
}
res.locals.username = username;
next();
}
app.head("/api/object/:name", requireDeviceToken, (req, res) => {
const name = req.params.name;
if (typeof name !== "string" || !isValidObjectName(name)) {
res.status(400).end();
return;
}
uploads
.exists(name)
.then((exists) => res.status(exists ? 204 : 404).end())
.catch(() => res.status(500).end());
});
app.put("/api/upload", requireDeviceToken, (req, res, next) => {
const name = req.query.name;
if (typeof name !== "string" || !isValidObjectName(name)) {
res.status(400).json({ error: "name must be <sha256 hex>.<extension>" });
return;
}
uploads
.receive(name, req)
.then((outcome) => {
switch (outcome.status) {
case "created":
console.log(`stored ${name} for ${res.locals.username}`);
res.status(201).json({ stored: name });
return;
case "exists":
res.status(200).json({ stored: name, existed: true });
return;
case "hash-mismatch":
res.status(409).json({ error: "content hash mismatch", actual: outcome.actual });
return;
case "too-large":
res.status(413).json({ error: "upload exceeds the configured maximum" });
return;
}
})
.catch(next);
});
// Uniform error surface; the details go to the log, not the client.
app.use((error: unknown, _req: Request, res: Response, _next: NextFunction) => {
console.error(error);
if (!res.headersSent) {
res.status(500).json({ error: "internal error" });
}
});
async function main(): Promise<void> {
await uploads.init();
app.listen(config.port, () => {
console.log(`internaluploader-server listening on :${config.port}`);
});
}
void main();

106
server/src/oidc.ts Normal file
View File

@@ -0,0 +1,106 @@
import type { IncomingMessage } from "node:http";
import { Issuer, generators, type Client } from "openid-client";
import type { Config } from "./config.ts";
let issuerPromise: Promise<Issuer<Client>> | undefined;
/** Discovery, memoised — but NOT on failure, or one early miss poisons the pod. */
export async function getIssuer(config: Config): Promise<Issuer<Client>> {
if (!issuerPromise) {
issuerPromise = Issuer.discover(config.oidc.issuer).catch((error: unknown) => {
issuerPromise = undefined;
throw error;
});
}
return issuerPromise;
}
export function redirectUri(config: Config): string {
return `${config.publicOrigin}/oidc/callback`;
}
async function clientFor(config: Config): Promise<Client> {
const issuer = await getIssuer(config);
return new issuer.Client({
client_id: config.oidc.clientId,
client_secret: config.oidc.clientSecret,
redirect_uris: [redirectUri(config)],
response_types: ["code"],
});
}
export interface AuthRequest {
url: string;
state: string;
codeVerifier: string;
}
export async function beginAuth(config: Config): Promise<AuthRequest> {
const client = await clientFor(config);
const state = generators.state();
const codeVerifier = generators.codeVerifier();
return {
// No `prompt: login` here, deliberately — unlike qr-login, an existing
// authentik session is exactly what we want to reuse: pairing is an
// authorization act, not a step-up authentication.
url: client.authorizationUrl({
scope: "openid email profile",
state,
code_challenge: generators.codeChallenge(codeVerifier),
code_challenge_method: "S256",
}),
state,
codeVerifier,
};
}
export interface Identity {
username: string;
groups: string[];
}
export async function completeAuth(
config: Config,
request: IncomingMessage,
expected: { state: string; codeVerifier: string },
): Promise<Identity> {
const client = await clientFor(config);
const uri = redirectUri(config);
const tokenSet = await client.callback(uri, client.callbackParams(request), {
state: expected.state,
code_verifier: expected.codeVerifier,
});
const claims = tokenSet.claims();
const groups = Array.isArray(claims.groups)
? claims.groups.filter((g): g is string => typeof g === "string")
: undefined;
// Fail closed, and say which failure it is. An absent claim means the
// provider is missing its scope mapping; treating that as "no groups" would
// lock everyone out silently, and as "allow" would let everyone in silently.
if (groups === undefined) {
throw new Error(
"ID token carries no `groups` claim — the authentik provider is missing " +
"the profile scope mapping, so membership cannot be checked",
);
}
const username =
typeof claims.preferred_username === "string" ? claims.preferred_username : undefined;
if (!username) {
throw new Error(
"ID token carries no `preferred_username` claim — uploads could not be " +
"attributed to a user",
);
}
return { username, groups };
}
export function isAuthorised(config: Config, identity: Identity): boolean {
return identity.groups.includes(config.oidc.requiredGroup);
}

73
server/src/pages.ts Normal file
View File

@@ -0,0 +1,73 @@
import { toDataURL } from "qrcode";
export interface QrPayload {
v: 1;
u: string;
c: string;
}
export function qrPayload(publicOrigin: string, code: string): string {
const payload: QrPayload = { v: 1, u: publicOrigin, c: code };
return JSON.stringify(payload);
}
/**
* The whole UI: one page with a QR code. Deliberately says nothing about
* where files go or how to reach the server — the QR carries everything the
* app needs, and this page is already behind SSO.
*/
export async function pairingPage(
username: string,
payload: string,
ttlSeconds: number,
): Promise<string> {
const qrDataUri = await toDataURL(payload, { margin: 1, width: 320 });
return `<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>InternalUploader — pair a device</title>
<style>
:root { color-scheme: light dark; }
body {
font-family: system-ui, sans-serif;
display: flex; flex-direction: column; align-items: center;
justify-content: center; min-height: 100vh; margin: 0; gap: 1rem;
background: Canvas; color: CanvasText; text-align: center; padding: 0 1rem;
}
img { border-radius: 8px; background: white; padding: 8px; }
small { opacity: .6 }
a.button {
display: inline-block; padding: .5rem 1rem; border-radius: 6px;
border: 1px solid currentColor; text-decoration: none; color: inherit;
}
</style>
</head>
<body>
<h1>Pair a device</h1>
<p>Signed in as <strong>${escapeHtml(username)}</strong>. Scan with the InternalUploader app.</p>
<img src="${qrDataUri}" alt="pairing QR code" width="320" height="320">
<small>Code is single-use and expires in ${Math.round(ttlSeconds / 60)} minutes.</small>
<a class="button" href="/">New code</a>
</body>
</html>`;
}
export function deniedPage(username: string, group: string): string {
return `<!doctype html>
<html lang="en"><head><meta charset="utf-8"><title>Not authorised</title></head>
<body style="font-family: system-ui, sans-serif; text-align: center; margin-top: 4rem">
<h1>Not authorised</h1>
<p><strong>${escapeHtml(username)}</strong> is not in the <strong>${escapeHtml(group)}</strong> group.</p>
</body></html>`;
}
function escapeHtml(value: string): string {
return value
.replaceAll("&", "&amp;")
.replaceAll("<", "&lt;")
.replaceAll(">", "&gt;")
.replaceAll('"', "&quot;");
}

126
server/src/tokens.ts Normal file
View File

@@ -0,0 +1,126 @@
import { createHash, randomBytes, timingSafeEqual } from "node:crypto";
import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
import { join } from "node:path";
/**
* Pairing codes and device tokens.
*
* Pairing codes are IN MEMORY on purpose: they live five minutes and are spent
* once, so surviving a restart buys nothing — and a restart invalidating any
* outstanding QR is the safe direction.
*
* Device tokens persist, HASHED. The stored file never contains anything a
* reader could replay; the plaintext token exists only in the pairing response
* and on the phone.
*/
export interface PairingCode {
code: string;
username: string;
expiresAt: number;
}
export interface DeviceRecord {
/** sha256 hex of the bearer token. */
tokenHash: string;
username: string;
createdAt: string;
/** Coarse (per-day) so routine uploads do not rewrite the file constantly. */
lastUsedDay: string;
}
function sha256Hex(value: string): string {
return createHash("sha256").update(value).digest("hex");
}
function today(): string {
return new Date().toISOString().slice(0, 10);
}
export class TokenStore {
private readonly file: string;
private readonly tmp: string;
private readonly pairingTtlMs: number;
private devices: DeviceRecord[] = [];
private pending = new Map<string, PairingCode>();
// No parameter properties: `node --test` runs this file under strip-only
// type stripping, which rejects them.
constructor(stateDir: string, pairingTtlMs: number) {
this.pairingTtlMs = pairingTtlMs;
mkdirSync(stateDir, { recursive: true });
this.file = join(stateDir, "devices.json");
this.tmp = join(stateDir, "devices.json.tmp");
try {
this.devices = JSON.parse(readFileSync(this.file, "utf8")) as DeviceRecord[];
} catch {
this.devices = [];
}
}
private persist(): void {
// Write-then-rename so a crash mid-write cannot truncate the live store.
writeFileSync(this.tmp, JSON.stringify(this.devices, null, 2));
renameSync(this.tmp, this.file);
}
/** Mint a single-use pairing code for a browser-authenticated user. */
issuePairingCode(username: string): PairingCode {
this.sweep();
const entry: PairingCode = {
code: randomBytes(32).toString("hex"),
username,
expiresAt: Date.now() + this.pairingTtlMs,
};
this.pending.set(entry.code, entry);
return entry;
}
/** Exchange a pairing code for a device token. Single use, TTL enforced. */
redeemPairingCode(code: string): { token: string; username: string } | undefined {
this.sweep();
const entry = this.pending.get(code);
if (!entry) {
return undefined;
}
this.pending.delete(entry.code);
const token = randomBytes(32).toString("base64url");
this.devices.push({
tokenHash: sha256Hex(token),
username: entry.username,
createdAt: new Date().toISOString(),
lastUsedDay: today(),
});
this.persist();
return { token, username: entry.username };
}
/** Bearer-token check. Returns the owning username, or undefined. */
authenticate(token: string): string | undefined {
const hash = Buffer.from(sha256Hex(token), "hex");
for (const device of this.devices) {
const stored = Buffer.from(device.tokenHash, "hex");
if (stored.length === hash.length && timingSafeEqual(stored, hash)) {
if (device.lastUsedDay !== today()) {
device.lastUsedDay = today();
this.persist();
}
return device.username;
}
}
return undefined;
}
get deviceCount(): number {
return this.devices.length;
}
private sweep(): void {
const now = Date.now();
for (const [code, entry] of this.pending) {
if (entry.expiresAt <= now) {
this.pending.delete(code);
}
}
}
}

118
server/src/upload.ts Normal file
View File

@@ -0,0 +1,118 @@
import { createHash, randomUUID } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, rename, stat, unlink } from "node:fs/promises";
import { join } from "node:path";
import type { Readable } from "node:stream";
/**
* `<64 hex sha256>` plus an optional short, lowercase, alphanumeric extension.
* Everything else is rejected before any filesystem call — the name is the only
* client-controlled thing that ever reaches a path.
*/
const NAME_PATTERN = /^[0-9a-f]{64}(\.[a-z0-9]{1,16})?$/;
export function isValidObjectName(name: string): boolean {
return NAME_PATTERN.test(name);
}
export function hashOf(name: string): string {
return name.slice(0, 64);
}
export type UploadOutcome =
| { status: "created" }
| { status: "exists" }
| { status: "hash-mismatch"; actual: string }
| { status: "too-large" };
export class UploadDir {
private readonly downloadsDir: string;
private readonly maxBytes: number;
private readonly tmpDir: string;
// No parameter properties: `node --test` runs this file under strip-only
// type stripping, which rejects them.
constructor(downloadsDir: string, maxBytes: number) {
this.downloadsDir = downloadsDir;
this.maxBytes = maxBytes;
// Same filesystem as the destination, or the final rename stops being
// atomic and a half-written file could be seen with its final name.
this.tmpDir = join(downloadsDir, ".internaluploader-tmp");
}
async init(): Promise<void> {
await mkdir(this.tmpDir, { recursive: true });
}
async exists(name: string): Promise<boolean> {
try {
await stat(join(this.downloadsDir, name));
return true;
} catch {
return false;
}
}
/**
* Stream a request body to disk, verify its sha256 matches the declared
* name, and only then rename it into the downloads directory. Idempotent:
* a name that already exists is reported without touching anything.
*/
async receive(name: string, body: Readable): Promise<UploadOutcome> {
if (await this.exists(name)) {
return { status: "exists" };
}
const tmpPath = join(this.tmpDir, randomUUID());
const hash = createHash("sha256");
let received = 0;
try {
await new Promise<void>((resolve, reject) => {
const out = createWriteStream(tmpPath, { flags: "wx", mode: 0o644 });
body.on("data", (chunk: Buffer) => {
received += chunk.length;
if (received > this.maxBytes) {
body.destroy();
out.destroy();
reject(new TooLargeError());
return;
}
hash.update(chunk);
});
body.on("error", reject);
out.on("error", reject);
out.on("finish", resolve);
body.pipe(out);
});
const actual = hash.digest("hex");
if (actual !== hashOf(name)) {
await unlink(tmpPath);
return { status: "hash-mismatch", actual };
}
// Lost race with a concurrent identical upload: same content, so the
// file already on disk is exactly what this request carried.
if (await this.exists(name)) {
await unlink(tmpPath);
return { status: "exists" };
}
await rename(tmpPath, join(this.downloadsDir, name));
return { status: "created" };
} catch (error) {
await unlink(tmpPath).catch(() => {});
if (error instanceof TooLargeError) {
return { status: "too-large" };
}
throw error;
}
}
}
class TooLargeError extends Error {
constructor() {
super("upload exceeds the configured maximum");
}
}

View File

@@ -0,0 +1,42 @@
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { readFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { test } from "node:test";
import { TokenStore } from "../src/tokens.ts";
test("pairing code round-trip mints a working device token", async () => {
const dir = mkdtempSync(join(tmpdir(), "tokens-"));
const store = new TokenStore(dir, 60_000);
const pairing = store.issuePairingCode("michal");
assert.match(pairing.code, /^[0-9a-f]{64}$/);
const redeemed = store.redeemPairingCode(pairing.code);
assert.ok(redeemed);
assert.equal(redeemed.username, "michal");
// Single use.
assert.equal(store.redeemPairingCode(pairing.code), undefined);
// The token authenticates, wrong tokens do not.
assert.equal(store.authenticate(redeemed.token), "michal");
assert.equal(store.authenticate("not-a-token"), undefined);
// Nothing on disk contains the plaintext token.
const persisted = await readFile(join(dir, "devices.json"), "utf8");
assert.ok(!persisted.includes(redeemed.token));
// A fresh store reads the same devices back.
const reloaded = new TokenStore(dir, 60_000);
assert.equal(reloaded.authenticate(redeemed.token), "michal");
});
test("expired pairing codes are refused", () => {
const dir = mkdtempSync(join(tmpdir(), "tokens-"));
const store = new TokenStore(dir, -1); // already expired at issue time
const pairing = store.issuePairingCode("michal");
assert.equal(store.redeemPairingCode(pairing.code), undefined);
});

View File

@@ -0,0 +1,72 @@
import assert from "node:assert/strict";
import { createHash, randomBytes } from "node:crypto";
import { mkdtempSync } from "node:fs";
import { readFile, readdir, stat } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { Readable } from "node:stream";
import { test } from "node:test";
import { UploadDir, hashOf, isValidObjectName } from "../src/upload.ts";
test("object name validation", () => {
const hash = "a".repeat(64);
assert.ok(isValidObjectName(hash));
assert.ok(isValidObjectName(`${hash}.mp4`));
assert.ok(isValidObjectName(`${hash}.webm`));
assert.ok(!isValidObjectName(`${hash}.`));
assert.ok(!isValidObjectName(`${hash}.MP4`), "uppercase extension");
assert.ok(!isValidObjectName(`${hash}.m p4`), "space in extension");
assert.ok(!isValidObjectName(`../../etc/passwd`), "traversal");
assert.ok(!isValidObjectName(`${"a".repeat(63)}.mp4`), "short hash");
assert.ok(!isValidObjectName(`${hash}.${"x".repeat(17)}`), "long extension");
assert.equal(hashOf(`${hash}.mp4`), hash);
});
function payload(bytes: Buffer): { name: string; body: Readable } {
const digest = createHash("sha256").update(bytes).digest("hex");
return { name: `${digest}.bin`, body: Readable.from([bytes]) };
}
test("receive stores a verified upload and is idempotent", async () => {
const dir = mkdtempSync(join(tmpdir(), "uploads-"));
const uploads = new UploadDir(dir, 1024 * 1024);
await uploads.init();
const bytes = randomBytes(4096);
const { name, body } = payload(bytes);
assert.deepEqual(await uploads.receive(name, body), { status: "created" });
assert.deepEqual(await readFile(join(dir, name)), bytes);
assert.ok(await uploads.exists(name));
const again = await uploads.receive(name, Readable.from([bytes]));
assert.deepEqual(again, { status: "exists" });
// The temp dir holds no leftovers either way.
assert.deepEqual(await readdir(join(dir, ".internaluploader-tmp")), []);
});
test("receive rejects a body that does not match its name", async () => {
const dir = mkdtempSync(join(tmpdir(), "uploads-"));
const uploads = new UploadDir(dir, 1024 * 1024);
await uploads.init();
const name = `${"0".repeat(64)}.bin`;
const outcome = await uploads.receive(name, Readable.from([randomBytes(16)]));
assert.equal(outcome.status, "hash-mismatch");
await assert.rejects(stat(join(dir, name)));
assert.deepEqual(await readdir(join(dir, ".internaluploader-tmp")), []);
});
test("receive enforces the size ceiling", async () => {
const dir = mkdtempSync(join(tmpdir(), "uploads-"));
const uploads = new UploadDir(dir, 1000);
await uploads.init();
const bytes = randomBytes(4096);
const { name, body } = payload(bytes);
const outcome = await uploads.receive(name, body);
assert.equal(outcome.status, "too-large");
assert.deepEqual(await readdir(join(dir, ".internaluploader-tmp")), []);
});

26
server/tsconfig.json Normal file
View File

@@ -0,0 +1,26 @@
{
"compilerOptions": {
"target": "ES2023",
"module": "commonjs",
"moduleResolution": "node",
"lib": ["ES2023"],
"outDir": "dist",
"rootDir": "src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"declaration": false,
"sourceMap": false,
// Relative imports carry `.ts` and TypeScript rewrites them to `.js` on
// emit. This is what lets the SAME source run two ways: compiled to
// dist/ for the image, and under Node's strip-only type stripping for
// `node --test`, which resolves imports by their real filename. Without it
// any src module that imports another src module is untestable.
"allowImportingTsExtensions": true,
"rewriteRelativeImportExtensions": true
},
"include": ["src/**/*.ts"],
"exclude": ["node_modules", "dist", "test"]
}

13
server/tsconfig.test.json Normal file
View File

@@ -0,0 +1,13 @@
{
// The tests run under Node's type stripping, which resolves imports by their
// real filename — hence the `.ts` extensions in the test sources. The build
// tsconfig cannot allow those, so tests get their own noEmit config.
// Without this the test files would be the one surface nothing checks.
"extends": "./tsconfig.json",
"compilerOptions": {
"noEmit": true,
"allowImportingTsExtensions": true,
"rootDir": "."
},
"include": ["src/**/*.ts", "test/**/*.ts"]
}