|
1 | | ---- |
2 | | -icon: location-exclamation |
3 | | -description: How to run a Bacalhau devstack locally |
4 | | ---- |
| 1 | +# Running Locally with Devstack |
5 | 2 |
|
6 | | -# Running Locally In Devstack |
| 3 | +Bacalhau's devstack provides a quick way to spin up a local testing environment with multiple nodes. This is perfect for development, testing, or simply exploring Bacalhau's capabilities without setting up a full production environment. |
7 | 4 |
|
8 | | -You can run a stand-alone Bacalhau network on your computer with the following guide. |
| 5 | +## Prerequisites |
9 | 6 |
|
10 | | -The `devstack` command of `bacalhau` will start a 4 node cluster with 3 compute and 1 requester nodes. |
| 7 | +- **Docker Engine**: Must be installed and running |
| 8 | +- **Bacalhau**: Latest version installed |
11 | 9 |
|
12 | | -This is useful to kick the tires and/or developing on the codebase. It's also the tool used by some tests. |
| 10 | +## Installation |
13 | 11 |
|
14 | | -## Pre-requisites |
15 | | - |
16 | | -1. `x86_64` or `ARM64` architecture |
17 | | - 1. Ubuntu 20.0+ has most often been used for development and testing |
18 | | -2. [Docker Engine](https://docs.docker.com/get-docker/) |
19 | | -3. Latest Bacalhau release |
20 | | - |
21 | | -## Install Bacalhau |
22 | | - |
23 | | -You can install the Bacalhau CLI by running this command in a terminal: |
| 12 | +If you haven't installed Bacalhau yet: |
24 | 13 |
|
25 | 14 | ```bash |
26 | 15 | curl -sL https://get.bacalhau.org/install.sh | bash |
27 | 16 | ``` |
28 | 17 |
|
29 | | -See the [installation guide](running-locally.md#install-bacalhau) for more installation options. |
| 18 | +## Starting the Devstack |
30 | 19 |
|
31 | | -## Start the cluster |
| 20 | +Starting a local devstack is as simple as: |
32 | 21 |
|
33 | 22 | ```bash |
34 | 23 | bacalhau devstack |
35 | 24 | ``` |
36 | 25 |
|
37 | | -This will start a 4 node Bacalhau cluster. |
| 26 | +When initialization completes, you'll see a confirmation message: |
38 | 27 |
|
39 | | -Once everything has started up - you will see output like the following: |
40 | | - |
41 | | -```bash |
| 28 | +``` |
42 | 29 | Devstack is ready! |
43 | 30 | No. of requester only nodes: 1 |
44 | 31 | No. of compute only nodes: 3 |
45 | 32 | No. of hybrid nodes: 0 |
46 | | -To use the devstack, run the following commands in your shell: |
47 | | - |
48 | | -export BACALHAU_API_HOST=0.0.0.0 |
49 | | -export BACALHAU_API_PORT=34217 |
50 | 33 | ``` |
51 | 34 |
|
52 | | -The message above contains the environment variables you need for a new terminal window. You can paste these into a new terminal so that `bacalhau` will use your local devstack. Execute the [bacalhau node list](/cli/node/list.md) command to see the devstack cluster structure: |
| 35 | +## Using Your Devstack |
| 36 | + |
| 37 | +By default, devstack configures the orchestrator to run on the standard port (1234), so you can immediately use it without additional settings. Open a new terminal window and test your connection: |
53 | 38 |
|
54 | 39 | ```bash |
55 | 40 | bacalhau node list |
56 | | - ID TYPE APPROVAL STATUS LABELS CPU MEMORY DISK GPU |
57 | | - node-0 Requester APPROVED CONNECTED Architecture=amd64 Operating-System=linux |
58 | | - env=devstack id=node-0 name=node-0 |
59 | | - node-1 Compute APPROVED CONNECTED Architecture=amd64 Operating-System=linux 1.4 / 2.7 GB / 13.1 GB / 0 / |
60 | | - env=devstack id=node-1 name=node-1 1.4 2.7 GB 13.1 GB 0 |
61 | | - node-2 Compute APPROVED CONNECTED Architecture=amd64 Operating-System=linux 1.4 / 2.7 GB / 13.1 GB / 0 / |
62 | | - env=devstack id=node-2 name=node-2 1.4 2.7 GB 13.1 GB 0 |
63 | | - node-3 Compute APPROVED CONNECTED Architecture=amd64 Operating-System=linux 1.4 / 2.7 GB / 13.1 GB / 0 / |
64 | | - env=devstack id=node-3 name=node-3 1.4 2.7 GB 13.1 GB 0 |
65 | 41 | ``` |
66 | 42 |
|
67 | | -## New Terminal Window |
68 | | - |
69 | | -Open an additional terminal window to be used for submitting jobs. Copy and paste environment variables from previous message into this window, e.g.: |
| 43 | +You should see the nodes in your devstack: |
70 | 44 |
|
71 | | -```bash |
72 | | -export BACALHAU_API_HOST=0.0.0.0 |
73 | | -export BACALHAU_API_PORT=34217 |
| 45 | +``` |
| 46 | +ID TYPE APPROVAL STATUS LABELS |
| 47 | +node-0 Requester APPROVED CONNECTED Architecture=amd64 Operating-System=linux |
| 48 | + env=devstack id=node-0 name=node-0 |
| 49 | +node-1 Compute APPROVED CONNECTED Architecture=amd64 Operating-System=linux |
| 50 | + env=devstack id=node-1 name=node-1 |
| 51 | +node-2 Compute APPROVED CONNECTED Architecture=amd64 Operating-System=linux |
| 52 | + env=devstack id=node-2 name=node-2 |
| 53 | +node-3 Compute APPROVED CONNECTED Architecture=amd64 Operating-System=linux |
| 54 | + env=devstack id=node-3 name=node-3 |
74 | 55 | ``` |
75 | 56 |
|
76 | | -You are now ready to submit a job to your local devstack. |
77 | | - |
78 | | -## Submit a simple job |
| 57 | +## Running a Simple Job |
79 | 58 |
|
80 | | -This will submit a simple job to a single node: |
| 59 | +Submit a simple job to test your devstack: |
81 | 60 |
|
82 | 61 | ```bash |
83 | | -bacalhau docker run \ |
84 | | ---publisher local \ |
85 | | -alpine echo "hello devstack test" |
| 62 | +bacalhau docker run alpine echo "hello devstack" |
86 | 63 | ``` |
87 | 64 |
|
88 | | -This should output something like the following: |
89 | | - |
90 | | -```bash |
91 | | -Job successfully submitted. Job ID: j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
92 | | -Checking job status... (Enter Ctrl+C to exit at any time, your job will continue running): |
| 65 | +Note that no publisher is needed for basic testing - Bacalhau will handle the job execution without storing results to any external location. |
93 | 66 |
|
94 | | - TIME EXEC. ID TOPIC EVENT |
95 | | - 11:06:24.789 Submission Job submitted |
96 | | - 11:06:24.798 e-4fba1f7c Scheduling Requested execution on node-1 |
97 | | - 11:06:24.828 e-4fba1f7c Execution Running |
98 | | - 11:06:25.173 e-4fba1f7c Execution Completed successfully |
| 67 | +## Advanced Configuration |
99 | 68 |
|
100 | | -To get more details about the run, execute: |
101 | | - bacalhau job describe j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
| 69 | +Devstack accepts the same configuration options as the `bacalhau serve` command. Here are some useful configuration examples: |
102 | 70 |
|
103 | | -To get more details about the run executions, execute: |
104 | | - bacalhau job executions j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
105 | | -To download the results, execute: |
106 | | - bacalhau job get j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
107 | | - |
108 | | -``` |
| 71 | +### Customizing Resource Limits |
109 | 72 |
|
110 | | -Use [bacalhau job describe](/cli/job/describe.md) command to view results: |
| 73 | +Limit the CPU and memory allocation for compute nodes: |
111 | 74 |
|
112 | 75 | ```bash |
113 | | -bacalhau job describe j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
| 76 | +bacalhau devstack -c Compute.AllocatedCapacity.CPU="50%" -c Compute.AllocatedCapacity.Memory="2Gi" |
114 | 77 | ``` |
115 | 78 |
|
116 | | -This should output info about job execution and results: |
| 79 | +### Configuring Job Defaults |
117 | 80 |
|
118 | | -```bash |
119 | | -ID = j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
120 | | -Name = j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
121 | | -Namespace = default |
122 | | -Type = batch |
123 | | -State = Completed |
124 | | -Count = 1 |
125 | | -Created Time = 2024-10-30 10:06:24 |
126 | | -Modified Time = 2024-10-30 10:06:25 |
127 | | -Version = 0 |
128 | | - |
129 | | -Summary |
130 | | -Completed = 1 |
131 | | - |
132 | | -Job History |
133 | | - TIME TOPIC EVENT |
134 | | - 2024-10-30 11:06:24 Submission Job submitted |
135 | | - 2024-10-30 11:06:24 State Update Running |
136 | | - 2024-10-30 11:06:25 State Update Completed |
137 | | - |
138 | | -Executions |
139 | | - ID NODE ID STATE DESIRED REV. CREATED MODIFIED COMMENT |
140 | | - e-4fba1f7c node-1 Completed Stopped 6 6m11s ago 6m11s ago |
141 | | - |
142 | | -Execution e-4fba1f7c History |
143 | | - TIME TOPIC EVENT |
144 | | - 2024-10-30 11:06:24 Scheduling Requested execution on node-1 |
145 | | - 2024-10-30 11:06:24 Execution Running |
146 | | - 2024-10-30 11:06:25 Execution Completed successfully |
147 | | - |
148 | | -Standard Output |
149 | | -hello devstack test |
150 | | -``` |
151 | | - |
152 | | -Use bacalhau job get command to download job results: |
| 81 | +Set default resource requirements for all batch jobs: |
153 | 82 |
|
154 | 83 | ```bash |
155 | | -bacalhau job get j-5b0ee6dd-6080-4277-99ce-fdf179907b25 |
| 84 | +bacalhau devstack -c JobDefaults.Batch.Task.Resources.CPU="250m" -c JobDefaults.Batch.Task.Resources.Memory="512Mb" |
156 | 85 | ``` |
157 | 86 |
|
158 | | -Results will be downloaded to the current directory. Job results should have the following structure: |
| 87 | +## Accessing the Web UI |
| 88 | + |
| 89 | +The Web UI is enabled by default in devstack for easier monitoring and management. Access it by opening your browser and navigating to: |
159 | 90 |
|
160 | 91 | ``` |
161 | | -j-5b0ee6dd/ |
162 | | -├── exitCode |
163 | | -├── stderr |
164 | | -└── stdout |
| 92 | +http://localhost:8438 |
165 | 93 | ``` |
166 | 94 |
|
167 | | -If you execute `cat stdout` it should read `hello devstack test`. If you write any files in your job, they will appear in `volumes/output`. |
| 95 | +The Web UI allows you to: |
| 96 | + |
| 97 | +- View all running and completed jobs |
| 98 | +- Inspect node status and resources |
| 99 | +- Monitor job execution details and logs |
168 | 100 |
|
169 | | -## Support |
| 101 | +## Stopping Devstack |
170 | 102 |
|
171 | | -If you have questions or need support or guidance, please reach out to the [Bacalhau team via Slack](https://bacalhauproject.slack.com/ssb/redirect) (**#general** channel). |
| 103 | +When you're done, simply press `Ctrl+C` in the terminal where devstack is running to shut everything down. |
0 commit comments