(IMPLEMENTATION) / Comprehensive Example Restoration for Ianvs(Robot) - Lifelong Learning Example #297

abhishek-8081 · 2025-11-26T19:44:29Z

LFX Mentorship 2025 Term 3: Complete Life-Long learning Example(Robot) Implementation for Ianvs
What type of PR is this?
example restoration

What this PR does / why we need it
This PR completes the full implementation of the life-long robot example in the Ianvs project as part of the LFX Mentorship 2025 Term 3.
All major components are complete (example code, tests, documentation).
The only remaining task is CI/CD integration.

Which issue(s) this PR fixes?
Fixes #287 #263 #230

@MooreZheng @hsj576

kubeedge-bot · 2025-11-26T19:44:43Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign moorezheng after the PR has been reviewed.
You can assign the PR to them by writing /assign @moorezheng in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: Abhishek Kumar <[email protected]>

gemini-code-assist · 2025-11-26T19:46:00Z

Summary of Changes

Hello @abhishek-8081, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request finalizes the comprehensive implementation of the lifelong learning robot example within the Ianvs project, completing a major milestone for the LFX Mentorship program. The changes focus on enhancing the robustness and efficiency of the core lifelong learning paradigm. This includes streamlining result handling during model evaluation, improving metric computation to adapt to different output scenarios, and updating various configuration files for a more standardized and explicit setup. The PR also reflects adjustments in the integration strategy for external models, suggesting a more modular approach to their use within the evaluation framework.

Highlights

Refined Lifelong Learning Paradigm Logic: Adjusted the run method in lifelong_learning.py to streamline result handling in 'no-inference' mode and modified the _train method for clearer environment variable setup during initial training rounds.
Improved Metric Computation Robustness: Enhanced TestCase.compute_metrics to gracefully handle scenarios where a direct paradigm_result is unavailable, instead relying on task_avg_acc from system metrics for accuracy calculation.
Streamlined Evaluation and Training Processes: Removed direct imports and usage of external models like SAM and Segformer from RFNet/eval.py, indicating a shift in their integration or evaluation strategy. Additionally, training visualization calls were commented out in RFNet/train.py to potentially reduce overhead.
Configuration Updates: Modified various YAML configuration files, including benchmarkingjob-simple.yaml, testenv-robot.yaml, and rfnet_algorithm-simple.yaml, to use absolute paths for datasets and modules, and adjusted module inclusions for a more standardized environment setup.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request restores a comprehensive lifelong learning example. The changes primarily involve refactoring the lifelong_learning paradigm to support a no-inference mode and updating various configuration files. My review has identified a few issues, mainly related to hardcoded absolute paths in YAML configuration files which affect the portability of the example. I have suggested using relative paths or generic paths as documented. Additionally, there's a logic bug in the _train method in lifelong_learning.py concerning environment variable settings that needs to be addressed.

gemini-code-assist · 2025-11-26T19:49:58Z

core/testcasecontroller/algorithm/paradigm/lifelong_learning/lifelong_learning.py

+                                                              self.dataset.test_url,
+                                                              "test")
+
+        return None, self.system_metric_info


The run method returns None for the test result in no-inference mode, which is inconsistent with other modes and the method's docstring. The evaluation result is available in the test_res variable from the my_eval call. It should be returned to be consistent.

Suggested change

return None, self.system_metric_info

return test_res, self.system_metric_info

gemini-code-assist · 2025-11-26T19:49:59Z

core/testcasecontroller/algorithm/paradigm/lifelong_learning/lifelong_learning.py

+        if rounds < 1:
+            os.environ["CLOUD_KB_INDEX"] = cloud_task_index
+            os.environ["OUTPUT_URL"] = train_output_dir
+        if rounds < 1:


The environment variables CLOUD_KB_INDEX and OUTPUT_URL are now set only if rounds < 1. This will cause issues in subsequent training rounds (rounds >= 1). Additionally, the if rounds < 1: check is duplicated. These variables should be set unconditionally before the if block.

os.environ["CLOUD_KB_INDEX"] = cloud_task_index os.environ["OUTPUT_URL"] = train_output_dir

gemini-code-assist · 2025-11-26T19:49:59Z

examples/robot/lifelong_learning_bench/semantic-segmentation/benchmarkingjob-simple.yaml

+  # job name of bechmarking; string type;
+  name: "benchmarkingjob"
+  # the url address of job workspace that will reserve the output of tests; string type;
+  workspace: "/home/abhishek/projects/kumar/ianvs/lifelong_learning_bench/robot-workspace-test"


The workspace path is hardcoded to an absolute path specific to a user's machine. This makes the example not portable. Please use a relative path.

workspace: "./lifelong_learning_bench/robot-workspace-test"

gemini-code-assist · 2025-11-26T19:49:59Z

examples/robot/lifelong_learning_bench/semantic-segmentation/benchmarkingjob-simple.yaml

+
+  # the url address of test environment configuration file; string type;
+  # the file format supports yaml/yml;
+  testenv: "/home/abhishek/projects/kumar/ianvs/examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/testenv-robot.yaml"


The testenv path is hardcoded to an absolute path. This makes the example not portable. Please use a relative path.

testenv: "./examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/testenv-robot.yaml"

gemini-code-assist · 2025-11-26T19:49:59Z

examples/robot/lifelong_learning_bench/semantic-segmentation/benchmarkingjob-simple.yaml

+      - name: "rfnet_lifelong_learning"
+        # the url address of test algorithm configuration file; string type;
+        # the file format supports yaml/yml
+        url: "/home/abhishek/projects/kumar/ianvs/examples/robot/lifelong_learning_bench/semantic-segmentation/testalgorithms/rfnet/rfnet_algorithm-simple.yaml"


The algorithm url is hardcoded to an absolute path. This makes the example not portable. Please use a relative path.

url: "./examples/robot/lifelong_learning_bench/semantic-segmentation/testalgorithms/rfnet/rfnet_algorithm-simple.yaml"

gemini-code-assist · 2025-11-26T19:49:59Z

examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/testenv-robot.yaml

+    train_index: "/home/abhishek/cloud-robotics/640x480/train-index.txt"
+    # the url address of test dataset index; string type;
+    test_index: "/home/abhishek/cloud-robotics/640x480/test-index.txt"


Hardcoded user-specific absolute paths are used for train_index and test_index. This makes the example not portable. Please use the generic data paths as documented (e.g., /data/datasets/...) or relative paths.

train_index: "/data/datasets/robot_dataset/train-index-mix.txt" # the url address of test dataset index; string type; test_index: "/data/datasets/robot_dataset/test-index.txt"

gemini-code-assist · 2025-11-26T19:50:00Z

examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/testenv-robot.yaml

+      # metric name; string type;
+      name: "accuracy"
+      # the url address of python file
+      url: "/home/abhishek/projects/kumar/ianvs/examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/accuracy.py"


The url for the model metric is a hardcoded absolute path. This makes the example not portable. Please use a relative path.

url: "./examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/accuracy.py"

gemini-code-assist · 2025-11-26T19:50:00Z

examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/testenv-robot.yaml

+      # metric name; string type;
+    - name: "accuracy"
+      # the url address of python file
+      url: "/home/abhishek/projects/kumar/ianvs/examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/accuracy.py"


The url for the accuracy metric is a hardcoded absolute path. This makes the example not portable. Please use a relative path.

url: "./examples/robot/lifelong_learning_bench/semantic-segmentation/testenv/accuracy.py"

gemini-code-assist · 2025-11-26T19:50:00Z

core/testcasecontroller/algorithm/paradigm/lifelong_learning/lifelong_learning.py

+            self.edge_task_index, tasks_detail, test_res = self.my_eval(self.cloud_task_index,
+                                                      self.dataset.test_url,
+                                                      r)
+            task_avg_score = {'accuracy':0.0}
+            i = 0
+            for detail in tasks_detail:
+                i += 1
+                scores = detail.scores
+                entry = detail.entry
+                LOGGER.info(f"{entry} scores: {scores}")
+                task_avg_score['accuracy'] += scores['accuracy']
+            task_avg_score['accuracy'] = task_avg_score['accuracy']/i
+            self.system_metric_info[SystemMetricType.TASK_AVG_ACC.value] = task_avg_score
+            LOGGER.info(task_avg_score)
+            # job = self.build_paradigm_job(ParadigmType.LIFELONG_LEARNING.value)
+            # inference_dataset = self.dataset.load_data(self.dataset.test_url, "eval",
+            #                                        feature_process=_data_feature_process)
+            # kwargs = {}
+            # test_res = job.my_inference(inference_dataset, **kwargs)
+            #del job


In no-inference mode, the final evaluation result test_res is computed but then None is returned by the run method. This contradicts the method's docstring which states it returns a numpy.ndarray. While the logic in testcase.py is adapted to handle None, this makes the code confusing as test_res becomes an unused variable. It's better to return test_res if it's the intended result, or rename it to _ if it's meant to be ignored.

gemini-code-assist · 2025-11-26T19:50:00Z

core/testcasecontroller/testcase/testcase.py

+                if paradigm_result is None:
+                    continue
+                metric_res[metric_name] = metric_func(test_dataset.y, paradigm_result)
+        if paradigm_result is None:
+            metric_res["accuracy"] = metric_res["task_avg_acc"]


This logic handles the case where paradigm_result is None by assigning task_avg_acc to accuracy. This seems to be a workaround for the no-inference mode. While it works, it makes the control flow a bit complex. A better approach might be to have the paradigm always return a consistent data structure, even if it's just the accuracy score, to avoid this special handling in the TestCase.

abhishek-8081 · 2025-11-26T20:22:37Z

Please review this.
@MooreZheng @hsj576

hsj576

Right now, in the GitHub “Files changed” view, all files under the lifelong learning example are marked as changed. Please fix this so that only the files where you actually modified code in the lifelong learning example are shown as changed.

kubeedge-bot requested review from MooreZheng and Poorunga November 26, 2025 19:44

kubeedge-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 26, 2025

lfx

e092acc

Signed-off-by: Abhishek Kumar <[email protected]>

abhishek-8081 force-pushed the restore-work branch from d79cf6b to e092acc Compare November 26, 2025 19:45

abhishek-8081 mentioned this pull request Nov 26, 2025

(IMPLEMENTATION) / Comprehensive Example Restoration for Ianvs - Lifelong Learning Example #296

Closed

gemini-code-assist bot reviewed Nov 26, 2025

View reviewed changes

hsj576 requested changes Nov 27, 2025

View reviewed changes

kubeedge-bot assigned hsj576 Nov 27, 2025

abhishek-8081 mentioned this pull request Nov 28, 2025

REQUEST: New membership for abhishek-8081 kubeedge/community#244

Open

5 tasks

	return None, self.system_metric_info
	return test_res, self.system_metric_info

(IMPLEMENTATION) / Comprehensive Example Restoration for Ianvs(Robot) - Lifelong Learning Example #297

Are you sure you want to change the base?

(IMPLEMENTATION) / Comprehensive Example Restoration for Ianvs(Robot) - Lifelong Learning Example #297

Uh oh!

Conversation

abhishek-8081 commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kubeedge-bot commented Nov 26, 2025

Uh oh!

gemini-code-assist bot commented Nov 26, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

abhishek-8081 commented Nov 26, 2025

Uh oh!

hsj576 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

abhishek-8081 commented Nov 26, 2025 •

edited

Loading