Bug Description

In router/batchrouter/handle.go, the upload() function opens a gzip file for uploading to object storage but never closes the file descriptor, regardless of whether the upload succeeds or fails.

Location

router/batchrouter/handle.go — function upload(), line 384

Code

outputFile, err := os.Open(gzipFilePath)
if err != nil {
    panic(err)
}
// ❌ no defer outputFile.Close() here

// ... ~70 lines of code ...

uploadOutput, err := uploader.Upload(context.TODO(), outputFile, keyPrefixes...)
if err != nil {
    return UploadResult{Error: err, ...}  // fd leaked
}

return UploadResult{...}  // fd also leaked on success

Impact

upload() is called on every batch job cycle for every active destination. Each call leaks one OS file descriptor. Under sustained load this will exhaust the process's fd limit (ulimit -n), causing new os.Open / socket calls to fail with too many open files — which surfaces as dropped events or a crashed process.

Expected Behaviour

The file descriptor should be closed after the upload completes (success or failure).

Suggested Fix

Add a defer immediately after the os.Open call:

outputFile, err := os.Open(gzipFilePath)
if err != nil {
    panic(err)
}
defer func() { _ = outputFile.Close() }()

This is already the correct pattern used in the sibling file archiver/worker.go (line 187):

file, err := os.Open(filePath)
if err != nil {
    return "", fmt.Errorf("open file %s: %w", filePath, err)
}
defer func() { _ = file.Close() }()

Environment

• File: router/batchrouter/handle.go
• Function: upload() (lines 240–483)
• Affected: all batch router destinations (warehouse staging + raw data uploads)