Skip to content

feat: add foundation models plugin with server management capabilities#7744

Merged
louis-jan merged 17 commits into
janhq:mainfrom
dev-miro26:feat/apple-foundation-models-provider
Mar 18, 2026
Merged

feat: add foundation models plugin with server management capabilities#7744
louis-jan merged 17 commits into
janhq:mainfrom
dev-miro26:feat/apple-foundation-models-provider

Conversation

@dev-miro26

Copy link
Copy Markdown
Contributor

New components

foundation-models-server/ — Swift CLI that wraps FoundationModels.framework
and exposes an OpenAI-compatible HTTP API (health, /v1/models, /v1/chat/completions).
Checks SystemLanguageModel.default.availability at startup and exits with a
descriptive error when Apple Intelligence is not available.

src-tauri/plugins/tauri-plugin-foundation-models/ — Rust Tauri plugin that
manages the server process lifecycle (spawn, readiness polling, graceful shutdown).
Returns structured error codes for all unavailability reasons
(DEVICE_NOT_ELIGIBLE, APPLE_INTELLIGENCE_NOT_ENABLED, MODEL_NOT_READY).

extensions/foundation-models-extension/ — Jan AIEngine extension. Exposes
the model as apple/on-device. Delegates streaming and non-streaming chat to the
local server over localhost, following the same HTTP proxy pattern as
mlx-extension.

Changes to existing files

File Change
src-tauri/Cargo.toml New foundation-models optional feature + dep
src-tauri/src/lib.rs Plugin registration + exit cleanup
src-tauri/capabilities/desktop.json foundation-models:default permission
src-tauri/tauri.macos.conf.json Bundle foundation-models-server binary
Makefile build-foundation-models-server + -if-exists targets with codesign
package.json Build script; exclude from Windows/Linux extension builds

Fixes Issues

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

### How to test

# 1. Build the server binary (requires Xcode 26 beta)
make build-foundation-models-server

# 2. Smoke-test the binary directly
./src-tauri/resources/bin/foundation-models-server --check
# expected: "[foundation-models] Foundation Models is available"

# 3. Start the server manually
./src-tauri/resources/bin/foundation-models-server --port 8765
# expected: "[foundation-models] server is listening on 127.0.0.1:8765"

# 4. Hit the health endpoint
curl http://localhost:8765/health
# expected: {"status":"ok"}

# 5. Send a chat request
curl -X POST http://localhost:8765/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"apple/on-device","messages":[{"role":"user","content":"Say hello"}]}'

# 6. Build Jan with the new extension + plugin
yarn build:foundation-models-server
make build-foundation-models-server-if-exists
yarn dev
# Then: load "Apple On-Device Model" from the model hub

@dev-miro26

Copy link
Copy Markdown
Contributor Author

Hi, @louis-jan
This is the PR. Could you please left your thought?
Thank you.

@louis-jan

Copy link
Copy Markdown
Contributor

@dev-miro26 is there a way to hide the provider when the device is "NOT_ELIGIBLE"?

@dev-miro26

Copy link
Copy Markdown
Contributor Author

@louis-jan
I understood what you want. I will fix it soon.

@dev-miro26 dev-miro26 force-pushed the feat/apple-foundation-models-provider branch from ab8d938 to 38e4574 Compare March 16, 2026 08:32
@dev-miro26

Copy link
Copy Markdown
Contributor Author

Hi, @louis-jan

Could you review this PR again?

@louis-jan

Copy link
Copy Markdown
Contributor

Sure. Will check

@louis-jan

louis-jan commented Mar 17, 2026

Copy link
Copy Markdown
Contributor

The app shows a blank page on launch.

Screenshot 2026-03-17 at 09 13 31

@dev-miro26

Copy link
Copy Markdown
Contributor Author

Hi, @louis-jan
Could you check again?
I have fixed. I understand there are some issues. Please leave your feedback again.

2 image
For this feature, need macos v26(tahoe).
Since yours is notEligible (not just disabled), this is a hardware limitation, it can't be fixed by toggling a setting. Apple Intelligence requires specific Apple Silicon models (M1 or later with sufficient RAM, and the device must not be a VM).
To check if you can enable it, go to System Settings > Apple Intelligence & Siri. If that section doesn't appear at all, your device genuinely doesn't support it.

@louis-jan

Copy link
Copy Markdown
Contributor

It looks great, I think we can polish it a bit. cc @Vanalite

@dev-miro26 dev-miro26 force-pushed the feat/apple-foundation-models-provider branch from f3bd4e1 to ed901fb Compare March 17, 2026 20:57
@dev-miro26

Copy link
Copy Markdown
Contributor Author

Hi, @louis-jan
Is this PR impossible to merge?
What features or codebase do I have to improve more?
Please leave your opinion kindly.

@louis-jan

Copy link
Copy Markdown
Contributor

@dev-miro26 @Vanalite should we polish the provider and model name a bit? The provider plugin / extension should have Product Name / icon?

@louis-jan louis-jan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@louis-jan

louis-jan commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

Will merge after CI gets passed

@dev-miro26

Copy link
Copy Markdown
Contributor Author

Hi, @louis-jan
All CI test was passed. Could you please merge this PR?
And let's improve this feature forward more together.

@louis-jan louis-jan merged commit d5aa7d4 into janhq:main Mar 18, 2026
17 checks passed
@github-project-automation github-project-automation Bot moved this to QA in Jan Mar 18, 2026
@louis-jan

louis-jan commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

@dev-miro26 gave it some more tests, seems like it could not perform a tool call, can you help take a look.

Screenshot 2026-03-18 at 13 52 50

Also it's a bit odd here.
Screenshot 2026-03-18 at 13 53 22

@dev-miro26

Copy link
Copy Markdown
Contributor Author

@louis-jan
Thanks for your feedback. If you don't mind could you send me current issues via recording video?
And Please create a new issues and assign to me.
I will works until this feature works perfectly with you.

@louis-jan

Copy link
Copy Markdown
Contributor

@dev-miro26 you can try tool use, see issues in the screenshot.

@louis-jan

Copy link
Copy Markdown
Contributor

@dev-miro26 Is it possible to use this to wrap the server in a Tauri plugin? So that we can get rid of the Apple Foundation model server in Swift.
https://lib.rs/crates/apple-foundation-models

@dev-miro26

Copy link
Copy Markdown
Contributor Author

@louis-jan
I htink, We can't fully get rid of Swift because Apple's FoundationModels framework only works with Swift, there's no way to call it from Rust directly. What do you think about this?

@louis-jan

Copy link
Copy Markdown
Contributor

@dev-miro26 can you check this https://lib.rs/crates/apple-foundation-models

@dev-miro26

Copy link
Copy Markdown
Contributor Author

@louis-jan
I understood what you want.
Current: Extension -> Tauri Plugin ->spawns Swift HTTP server -> FoundationModels
Goal: Extension ->Tauri Plugin (with apple-foundation-models) ->FoundationModels

Is this right?

@louis-jan

Copy link
Copy Markdown
Contributor

Yes! Exactly

@dev-miro26

Copy link
Copy Markdown
Contributor Author

@louis-jan
No problem, Could you make an new issue and assign it to me?
Let me make this feature perfectly.

@dev-miro26

Copy link
Copy Markdown
Contributor Author

@louis-jan
I have updated to use the apple-foundation-models.
May I open a new PR for this update?

@louis-jan

Copy link
Copy Markdown
Contributor

Yes please @dev-miro26 thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: QA

Development

Successfully merging this pull request may close these issues.

idea: Support for apple foundation model

2 participants