Skip to content

Desktop, Mobile: Resolves #15081: Speed up app startup by skipping unnecessary plugin file processing - #15085

Merged
laurent22 merged 7 commits into
devfrom
plugin_extraction_state
Apr 30, 2026
Merged

Desktop, Mobile: Resolves #15081: Speed up app startup by skipping unnecessary plugin file processing#15085
laurent22 merged 7 commits into
devfrom
plugin_extraction_state

Conversation

@laurent22

@laurent22 laurent22 commented Apr 13, 2026

Copy link
Copy Markdown
Owner

Don't extract plugin files if the files are already present. Keep track of what is available using an index file.

Partially addresses #12301

@coderabbitai

coderabbitai Bot commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 74e741e1-62ec-4957-93b0-178a19bbe84d

📥 Commits

Reviewing files that changed from the base of the PR and between f418390 and 2d7f9b8.

📒 Files selected for processing (2)
  • packages/lib/services/plugins/PluginService.ts
  • packages/lib/services/plugins/loadPlugins.test.ts

📝 Walkthrough

Walkthrough

Adds a persisted per-plugin extraction-state cache and switches JPL cache validation from MD5 to file size + mtime checks. Plugin extraction is skipped when the cache is valid; otherwise the package is re-extracted, manifest reloaded and cache updated. A test verifies skip/re-extract behaviour.

Changes

Cohort / File(s) Summary
Plugin extraction caching
packages/lib/services/plugins/PluginService.ts
Persist plugin-extraction-state.json, maintain in-memory extractionStates_, validate cached unpack by fs.stat() (size + mtime) and presence of manifest.json (and index.js when not manifest-only). Skip extraction on valid cache; re-extract, reload manifest and update cache on invalid.
Cache validation test
packages/lib/services/plugins/loadPlugins.test.ts
Adds test that spies on shim.fsDriver().tarExtract to assert extraction occurs on first load, is skipped on unload/reload when artifact unchanged, and re-occurs after recreating the packed JPL with modified content.
Test plugin generator
packages/lib/testing/plugins/createTestPlugin.ts
createTestPlugin gains `format?: 'js'

Sequence Diagram(s)

sequenceDiagram
    participant PS as PluginService
    participant FS as Filesystem
    participant TAR as TarExtractor
    participant MAN as ManifestLoader

    PS->>FS: stat(plugin.jpl) → {size, mtime}
    PS->>PS: lookup extractionStates_[fname]
    alt cache valid (size+mtime match && manifest(+index.js) exist)
        PS-->>PS: skip extraction, use unpack dir
    else cache missing/invalid
        PS->>FS: remove/create unpack dir
        PS->>TAR: tarExtract(plugin.jpl → unpack dir)
        TAR-->>PS: extraction complete
        PS->>MAN: load unpack/manifest.json
        MAN-->>PS: manifest loaded
        PS->>FS: write plugin-extraction-state.json (size, timestamp)
    end
Loading

Possibly related PRs

Suggested labels

plugins, performance


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name Status Explanation Resolution
Pr Description Must Follow Guidelines ❌ Error PR description lacks required sections: no explicit Test Plan explaining verification, no AI Assistance Disclosure statement, and problem statement focuses on implementation rather than user-visible impact. Add distinct Problem, Solution, and Test Plan sections following Joplin guidelines, document the test case, and include an AI Assistance Disclosure statement to achieve compliance.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately reflects the main change: implementing cache validation to skip unnecessary plugin extraction during startup, directly addressing the performance objective.
Description check ✅ Passed The description is related to the changeset, explaining the core approach of tracking extracted files via an index file to avoid re-extraction.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch plugin_extraction_state

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot added enhancement Feature requests and code enhancements plugins Anything related to Joplin's plugin system performance Performance issues labels Apr 13, 2026
@laurent22 laurent22 added the v3.7 label Apr 13, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
packages/lib/testing/plugins/createTestPlugin.ts (1)

33-41: Avoid shared temp build paths for packed plugin creation.

Line 34 uses a deterministic folder (plugin-build-${manifest.id}), which can collide in parallel test workers and make this helper flaky. Please generate a unique temp folder per call and clean it up after tarCreate.

Suggested hardening
 	if (format === 'jpl') {
-		const tempDir = join(Setting.value('tempDir'), `plugin-build-${manifest.id}`);
-		await mkdirp(tempDir);
-		await writeFile(join(tempDir, 'manifest.json'), JSON.stringify(manifest), 'utf-8');
-		await writeFile(join(tempDir, 'index.js'), scriptSource, 'utf-8');
-
-		const jplPath = join(Setting.value('pluginDir'), `${manifest.id}.jpl`);
-		await shim.fsDriver().tarCreate({ cwd: tempDir, file: jplPath }, ['manifest.json', 'index.js']);
+		const tempDir = join(
+			Setting.value('tempDir'),
+			`plugin-build-${manifest.id}-${Date.now()}-${Math.random().toString(36).slice(2)}`,
+		);
+		await mkdirp(tempDir);
+		try {
+			await writeFile(join(tempDir, 'manifest.json'), JSON.stringify(manifest), 'utf-8');
+			await writeFile(join(tempDir, 'index.js'), scriptSource, 'utf-8');
+
+			const jplPath = join(Setting.value('pluginDir'), `${manifest.id}.jpl`);
+			await shim.fsDriver().tarCreate({ cwd: tempDir, file: jplPath }, ['manifest.json', 'index.js']);
+		} finally {
+			await shim.fsDriver().remove(tempDir);
+		}
 	} else {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/lib/testing/plugins/createTestPlugin.ts` around lines 33 - 41, In
createTestPlugin, avoid the shared deterministic temp folder used for packing
(the tempDir built from `plugin-build-${manifest.id}`); instead create a unique
per-call temporary directory (e.g., use a random/suffixed name or system
mkdtemp) before calling `mkdirp`/writing `manifest.json` and `index.js`, use
that path for `shim.fsDriver().tarCreate` to produce `jplPath`, and then
remove/cleanup the temp directory after `tarCreate` completes (or in a finally
block) to prevent parallel-test collisions and leaks.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/lib/services/plugins/PluginService.ts`:
- Around line 321-333: The extraction validity check in PluginService (variables
unpackDir, manifestFilePath, extractionStates, extractionState, extractionValid)
only verifies manifest.json; update extractionValid to also verify the presence
(and optionally integrity) of the plugin entry script (e.g.,
`${unpackDir}/index.js` or the actual script name expected by the plugin
manifest) by checking shim.fsDriver().exists for that script before skipping
extraction; ensure the existence check uses the same unpackDir and include the
script path variable in the condition so extraction is only skipped when both
manifest and the plugin script are present.
- Around line 222-225: The saveExtractionStates method currently lets write
errors bubble up and block plugin startup; modify saveExtractionStates (and its
use of this.extractionStates_, extractionStatePath() and
shim.fsDriver().writeFile) to catch any errors from writeFile, set
this.extractionStates_ as before, attempt the async write inside a try/catch,
and on failure log the error (do not rethrow) so the Promise resolves normally
and plugin loading can continue; ensure the function signature and behavior
remain Promise<void> but never rejects due to cache write failures.

---

Nitpick comments:
In `@packages/lib/testing/plugins/createTestPlugin.ts`:
- Around line 33-41: In createTestPlugin, avoid the shared deterministic temp
folder used for packing (the tempDir built from `plugin-build-${manifest.id}`);
instead create a unique per-call temporary directory (e.g., use a
random/suffixed name or system mkdtemp) before calling `mkdirp`/writing
`manifest.json` and `index.js`, use that path for `shim.fsDriver().tarCreate` to
produce `jplPath`, and then remove/cleanup the temp directory after `tarCreate`
completes (or in a finally block) to prevent parallel-test collisions and leaks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9c385204-6bcb-4084-85a7-365f1e622ea6

📥 Commits

Reviewing files that changed from the base of the PR and between 1bbd603 and fe8b81b.

📒 Files selected for processing (3)
  • packages/lib/services/plugins/PluginService.ts
  • packages/lib/services/plugins/loadPlugins.test.ts
  • packages/lib/testing/plugins/createTestPlugin.ts

Comment thread packages/lib/services/plugins/PluginService.ts
Comment thread packages/lib/services/plugins/PluginService.ts
@laurent22
laurent22 force-pushed the plugin_extraction_state branch from 474aca0 to fe8b81b Compare April 14, 2026 09:59
@laurent22 laurent22 changed the title Desktop, Mobile: Speed up app startup by skipping unnecessary plugin file processing Desktop, Mobile: Resolves #15081: Speed up app startup by skipping unnecessary plugin file processing Apr 14, 2026
@coderabbitai coderabbitai Bot removed enhancement Feature requests and code enhancements plugins Anything related to Joplin's plugin system performance Performance issues labels Apr 14, 2026
@laurent22
laurent22 merged commit 22eb606 into dev Apr 30, 2026
16 of 22 checks passed
@laurent22
laurent22 deleted the plugin_extraction_state branch April 30, 2026 15:08
@coderabbitai coderabbitai Bot added plugins Anything related to Joplin's plugin system performance Performance issues labels Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance issues plugins Anything related to Joplin's plugin system v3.7

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant