Set up and run Azure Foundry Native LLM server & Open WebUI on Home windows Server 2025

Foundry Native is an on-device AI inference resolution providing efficiency, privateness, customization, and value benefits. It integrates seamlessly into your present workflows and purposes via an intuitive CLI, SDK, and REST API. Foundry Native has the next advantages:

• On-Machine Inference: Run fashions domestically by yourself {hardware}, lowering your prices whereas preserving all of your information in your machine.
• Mannequin Customization: Choose from preset fashions or use your individual to satisfy particular necessities and use circumstances.
• Value Effectivity: Get rid of recurring cloud service prices through the use of your present {hardware}, making AI extra accessible.
• Seamless Integration: Join along with your purposes via an SDK, API endpoints, or the CLI, with simple scaling to Azure AI Foundry as your wants develop.

Foundry Native is good for eventualities the place:

• You need to maintain delicate information in your machine.
• You should function in environments with restricted or no web connectivity.
• You need to cut back cloud inference prices.
• You want low-latency AI responses for real-time purposes.
• You need to experiment with AI fashions earlier than deploying to a cloud surroundings.

You may set up Foundry Native by operating the next command:

winget set up Microsoft.FoundryLocal

As soon as Foundry Native is put in, you obtain and work together with a mannequin from the command line through the use of a command like:

foundry mannequin run phi-4

This may obtain the phi-4 mannequin and supply a textual content primarily based chat interface. If you wish to work together with Foundry Native via an internet chat interface, you need to use the open supply Open WebUI challenge. You may set up Open WebUI on Home windows Server by performing the next steps:

Obtain OpenWebUIInstaller.exe from https://github.com/BrainDriveAI/OpenWebUI_CondaInstaller/releases. You will get warning messages from Home windows Defender SmartScreen. Copy OpenWebUIInstaller.exe into C:Temp.

In an elevated command immediate, run the next instructions

winget set up -e --id Anaconda.Miniconda3 --scope machine
$env:Path="C:ProgramDataminiconda3;" + $env:Path
$env:Path="C:ProgramDataminiconda3Scripts;" + $env:Path
$env:Path="C:ProgramDataminiconda3Librarybin;" + $env:Path
conda.exe tos settle for --override-channels --channel https://repo.anaconda.com/pkgs/major
conda.exe tos settle for --override-channels --channel https://repo.anaconda.com/pkgs/r
conda.exe tos settle for --override-channels --channel https://repo.anaconda.com/pkgs/msys2
C:TempOpenWebUIInstaller.exe

Then from the dialog select to put in and run Open WebUI. You then must take a number of further steps to configure Open WebUI to hook up with the Foundry Native endpoint.

1. Allow Direct Connections in Open WebUI
   1. Choose Settings and Admin Settings within the profile menu.
   2. Choose Connections within the navigation menu.
   3. Allow Direct Connections by turning on the toggle. This enables customers to hook up with their very own OpenAI suitable API endpoints.
2. Join Open WebUI to Foundry Native:
   1. Choose Settings within the profile menu.
   2. Choose Connections within the navigation menu.
   3. Choose + by Handle Direct Connections.
   4. For the URL, enter http://localhost:PORT/v1 the place PORT is the Foundry Native endpoint port (use the CLI command foundry service standing to search out it). Notice that Foundry Native dynamically assigns a port, so it is not at all times the identical.
   5. For the Auth, choose None.
   6. Choose Save

➡️ What’s Foundry Native https://study.microsoft.com/en-us/azure/ai-foundry/foundry-local/what-is-foundry-local

➡️ Open WebUI: https://docs.openwebui.com/