Create an experiment with a test dataset

Oct 13, 2025 by leolavanderia.com.br 74 views

BUG: While running experiments with multiple models, failed models don't show logs

As a user of Lumigator, a powerful tool for running experiments with multiple models, you expect to see logs for failed models to understand what went wrong. However, in some cases, these logs may not be visible, making it challenging to diagnose and fix issues. In this article, we will delve into the issue of failed models not showing logs in Lumigator and explore possible solutions.

In a preset environment, I was using Lumigator and trying to run an experiment with a sample dataset available for the use case of summarization. I chose multiple models for the test run and Mistral always kept on failing. Even with the failed model, I couldn't visualize any logs related to the execution failure.

To reproduce this issue, follow these steps:

Create an experiment with a test dataset: For the current scenario, it was the sample dialogsum dataset.
Select the use case: The use case "summarization" was the preselected one.
Choose multiple models to execute: I chose more than one model, including Mistral-7B-Instruct-v0.2.
Run the experiment: Once the experiment was run, after some time, it showed Status: Incomplete because one or more models failed.
Check the logs for failed models: For the failed models, when you click the logs from the Status section on the right-hand panel, it pops up an experimental logs window at the bottom, but it's EMPTY.
Verify the experiment status: Another issue is that though the experiment tab shows the experiment name and status as Incomplete, but when you click the experiment, and the right-hand vertical panel opens up, it says Status: Succeeded, which is wrong.

Unfortunately, there is no relevant log output provided in this case.

I would expect logs to show up for the failed execution for the particular model. Additionally, if the experiment is incomplete, then the status on the right-hand panel should mark as partially succeeded rather than Succeeded.

Here are the system details:

Lumigator version: Pre-installed version during a tech event
System: MacOS Sequoia 15.3.1
Browser: Firefox 135.0

Yes, I have searched for similar issues before submitting this one.

To resolve this issue, you can try the following:

Check the experiment settings: Ensure that the experiment settings are correct, and the models are properly configured.
Verify the dataset: Check if the dataset is correct and properly formatted.
Check the model performance: Monitor the model performance and adjust the hyperparameters as needed.
Check the logs: Check the logs for any errors or warnings that may indicate the cause of the failure.
Contact support: If none of the above steps resolve the issue, contact Lumigator support for further assistance.

In conclusion, failed models not showing logs in Lumigator can be a frustrating issue. By following the reproduction steps and checking the system information, you can identify the possible causes of the issue. Try the possible solutions to resolve the issue, and if none of them work, contact Lumigator support for further assistance.

To improve the user experience, Lumigator can consider the following:

Improve log visibility: Make it easier for users to access and view logs for failed models.
Enhance experiment status: Provide a more accurate status for experiments that are incomplete or failed.
Provide better error messages: Offer more informative error messages to help users diagnose and fix issues.

By addressing these issues, Lumigator can provide a better experience for its users and help them achieve their goals more efficiently.
Q&A: Resolving Issues with Failed Models in Lumigator

In our previous article, we discussed the issue of failed models not showing logs in Lumigator. We also provided possible solutions to resolve this issue. In this article, we will answer some frequently asked questions (FAQs) related to this issue.

A: The common causes of failed models in Lumigator include:

Incorrect experiment settings: Ensure that the experiment settings are correct, and the models are properly configured.
Incorrect dataset: Check if the dataset is correct and properly formatted.
Model performance issues: Monitor the model performance and adjust the hyperparameters as needed.
Log issues: Check the logs for any errors or warnings that may indicate the cause of the failure.

A: To troubleshoot failed models in Lumigator, follow these steps:

Check the experiment settings: Ensure that the experiment settings are correct, and the models are properly configured.
Verify the dataset: Check if the dataset is correct and properly formatted.
Check the model performance: Monitor the model performance and adjust the hyperparameters as needed.
Check the logs: Check the logs for any errors or warnings that may indicate the cause of the failure.
Contact support: If none of the above steps resolve the issue, contact Lumigator support for further assistance.

A: The logs for failed models may not be visible in Lumigator due to various reasons, including:

Log configuration issues: Ensure that the log configuration is correct, and the logs are properly enabled.
Log file issues: Check if the log file is correct and properly formatted.
Log viewer issues: Check if the log viewer is working correctly.

A: To improve the log visibility in Lumigator, follow these steps:

Check the log configuration: Ensure that the log configuration is correct, and the logs are properly enabled.
Verify the log file: Check if the log file is correct and properly formatted.
Check the log viewer: Check if the log viewer is working correctly.
Contact support: If none of the above steps resolve the issue, contact Lumigator support for further assistance.

A: The benefits of using Lumigator for model training and testing include:

Improved model performance: Lumigator provides a range of tools and features to improve model performance.
Enhanced log visibility: Lumigator provides a clear and concise log viewer to help users diagnose and fix issues.
Better experiment management: Lumigator provides a range of tools and features to manage experiments, including experiment tracking and version control.

A: To get help with Lumigator, follow these steps:

Check the documentation: Check the Lumigator documentation for answers to common questions.
Contact support: Contact Lumigator support for further assistance.
Join the community: Join the Lumigator community to connect with other users and get help from experts.

By following these FAQs, you can resolve issues with failed models in Lumigator and improve your overall experience with the platform.