Semantic similarity #13
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 30 commits
October 15, 2025 10:46
added 12 commits
October 23, 2025 11:51
anmarques
requested changes
Oct 30, 2025
Member
anmarques
left a comment
anmarques
left a comment
There was a problem hiding this comment.
Overall it looks good. It needs some small fixes
| config_semantic_similarity_args.update(semantic_similarity_args) | ||
| self.semantic_similarity_args = config_semantic_similarity_args | ||
|
|
||
| self.num_samples_per_dataset = config_kwargs.pop("num_samples_per_dataset", num_samples_per_dataset) |
Member
There was a problem hiding this comment.
This implementation allows arguments set in config to overwrite arguments passed in by the user. This should not be the case. Config arguments should follow one of these two options:
- They can be overridden by the user. In this case the use-provided value should be used.
- They can't be overridden by the user. If the user provides a value for an argument that can't be overridden the argument parsing should raise an error.
Collaborator
Author
There was a problem hiding this comment.
fair point, will fix
src/automation/tasks/scripts/semantic_similarity_generate_script.py
Outdated
Show resolved
Hide resolved
|
|
||
| try: | ||
| print(">>> Initializing vLLM...") | ||
| llm = LLM( |
Member
There was a problem hiding this comment.
Why are we using the LLM class instead of vllm serve?
Collaborator
Author
There was a problem hiding this comment.
The main branch has an old src/automation/vllm/server.py file the class VLLMServer but other branches use start_vllm_server`.
Also shouldn't the output of the LLM class be identical to the vllm serve api endpoint?
src/automation/tasks/scripts/semantic_similarity_generate_script.py
Outdated
Show resolved
Hide resolved
src/automation/tasks/scripts/semantic_similarity_score_script.py
Outdated
Show resolved
Hide resolved
added 11 commits
October 31, 2025 10:56
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.