Langchain #216

batwood-1 · 2024-12-09T14:52:44Z

Description

Related Issue(s)

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

I have added tests to cover my changes.
All new and existing tests passed.
My code follows the code style of this project.

…into langchain Merging evaluation pipeline together

…langchain Updating the LLM setup, as well as new tests and protocols

…langchain Conflicts: pyproject.toml

fabiocat93

hi @batwood-1 , i have left some comments. I feel some changes are needed here. You can start addressing my comments and we can meet early Jan and discuss this merge in person. what do you think?

requirements.txt

.pre-commit-config.yaml

pyproject.toml

src/senselab/text/tasks/evaluate_conversation/deep_eval.py

fabiocat93 · 2025-01-09T23:11:50Z

@batwood-1 thank you for addressing some of my comments. We will check this deeply tomorrow in person

…langchain

pyproject.toml

…hain

batwood-1

Fixed all comments, check out latest changes under 'fixed evaluations'

fabiocat93 · 2025-02-10T13:44:59Z

pyproject.toml

 scikit-learn = "~=1.5"
 nltk = "~=3.9"
+sacrebleu = "^2.4.3"
+pytest-testmon = "^2.1.1"


I suspect you don't want pytest-testmon in production, but maybe only among the dev dependencies. and why do you want this?

fabiocat93 · 2025-02-10T13:46:07Z

pyproject.toml

 umap-learn = "~=0.5"
 scikit-learn = "~=1.5"
 nltk = "~=3.9"
+sacrebleu = "^2.4.3"


For consistency and easier dependency solving, can you use the tilde as for the other dependencies? ~=2.4

fabiocat93 · 2025-02-10T13:46:20Z

pyproject.toml

 vocos = "~=0.1"
+deepeval = "~=2.2.6"
+rouge-score = "~=0.1.2"
+textstat = "^0.7.4"


same as above

fabiocat93 · 2025-02-10T13:47:12Z

pyproject.toml

 select = ["ANN", "D", "E", "F", "I"]
-ignore = [
-  "ANN101",  # self should not be annotated.
-  "ANN102"  # cls should not be annotated.


why this? I suspect ANN101 and ANN102 should be there since we never annotate self

fabiocat93 · 2025-02-10T13:49:18Z

tutorials/llms.ipynb

this is actually empty

fabiocat93 · 2025-02-10T14:02:59Z

src/senselab/text/tasks/llms/llm.py

+class LLM:
+    """Wrapper for invoking various LLMs.
+
+    This class provides a unified interface for interacting with LLMs,


is the external server the easiest way to interact with these models? sounds like an extra effort that we are asking to our users

fabiocat93 · 2025-02-10T14:04:07Z

src/senselab/text/tasks/llms/process_transcript_example.py

I feel this is more code for the ALI project than code that will should be part of a reusable API

fabiocat93 · 2025-02-10T14:05:35Z

src/senselab/utils/data_structures/llm_response.py

Why cannot llm response be a scriptline?

fabiocat93 · 2025-02-10T14:07:07Z

src/senselab/utils/data_structures/transcript_input.py

how much of this and transcript_output is ALI -specific or reusable by others? My general comment is to try split the code that is specific to ALI and the code that you want to make available to everyone in the community

fabiocat93

I have left comments here and there. I feel that we are going in the right direction, but you may want to think about a more generalizable API for interacting with the llms (not just in the context of the ALI project)

Bruce Atwood and others added 18 commits July 18, 2024 14:36

initial llm changes

c8a05de

deep eval test

5677fb7

test

7e48531

more selective changes

d3d6144

Merge branch 'main' into langchain

eed4434

Merge branch 'main' into langchain

0ca5d57

Merge branch 'main' into langchain

38dac0f

adjusting doc and setting up tutorial for llms + fixing style issues

a25d33e

Merge branch 'main' into deep_eval_test

626000e

Merge branch 'deep_eval_test' of https://github.com/sensein/senselab …

4d9a90a

…into langchain Merging evaluation pipeline together

Merge branch 'main' into langchain

9d54f24

updates

78ffb03

Merge branch 'langchain' of https://github.com/sensein/senselab into …

e581c07

…langchain Updating the LLM setup, as well as new tests and protocols

Merge branch 'main' into langchain

396133a

updated pyproject.toml

0fe4dd6

Merge branch 'langchain' of https://github.com/sensein/senselab into …

29aceba

…langchain Conflicts: pyproject.toml

updated pyproject.toml

74cedbc

update

143bb92

batwood-1 requested review from fabiocat93 and removed request for fabiocat93 December 9, 2024 14:56

batwood-1 self-assigned this Dec 9, 2024

batwood-1 added 2 commits December 10, 2024 11:40

updated lllms

3bbc485

Merge branch 'main' into langchain

f2c5a49

fabiocat93 requested changes Dec 23, 2024

View reviewed changes

fabiocat93 linked an issue Dec 23, 2024 that may be closed by this pull request

Integrate LLMs (with langchain) for conversational script analysis #227

Open

fabiocat93 added the enhancement New feature or request label Dec 23, 2024

fabiocat93 marked this pull request as draft December 23, 2024 16:53

fabiocat93 and others added 2 commits January 22, 2025 09:53

fixing pyproject.toml #3

599ea87

Merge branch 'main' into langchain

f9399c4

fabiocat93 mentioned this pull request Jan 22, 2025

testing deep eval #126

Closed

batwood-1 added 3 commits January 23, 2025 04:03

removed requirements.txt

c728b95

Merge branch 'langchain' of https://github.com/sensein/senselab into …

cad601f

…langchain

updated data structures

2ffdefd

fabiocat93 reviewed Jan 23, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

batwood-1 added 2 commits January 28, 2025 11:36

Merge branch 'main' of https://github.com/sensein/senselab into langc…

c05c335

…hain

fixed evaluations

71c76f4

batwood-1 commented Jan 30, 2025

View reviewed changes

batwood-1 marked this pull request as ready for review January 30, 2025 01:26

batwood-1 requested a review from fabiocat93 January 30, 2025 01:28

re-enabled dynamic versioning

6bc5d7c

fabiocat93 reviewed Feb 10, 2025

View reviewed changes

pyproject.toml Outdated

vocos = "~=0.1"

deepeval = "~=2.2.6"

rouge-score = "~=0.1.2"

textstat = "^0.7.4"

Copy link

Collaborator

fabiocat93 Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

fabiocat93 reviewed Feb 10, 2025

View reviewed changes

tutorials/llms.ipynb Outdated

Copy link

Collaborator

fabiocat93 Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is actually empty

fabiocat93 reviewed Feb 10, 2025

View reviewed changes

src/senselab/utils/data_structures/llm_response.py Outdated

Copy link

Collaborator

fabiocat93 Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why cannot llm response be a scriptline?

fabiocat93 reviewed Feb 10, 2025

View reviewed changes

updated pyproject.toml, removed llm_response

c33f2b0

fabiocat93 mentioned this pull request Apr 11, 2025

Llm evaluation #117

Closed

fabiocat93 marked this pull request as draft August 29, 2025 13:30

batwood-1 added 2 commits September 1, 2025 13:20

fixed pyproject.toml

12747e9

minor refactor

ef4c185

Langchain #216

Are you sure you want to change the base?

Langchain #216

Uh oh!

Conversation

batwood-1 commented Dec 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Uh oh!

fabiocat93 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fabiocat93 commented Jan 9, 2025

Uh oh!

Uh oh!

batwood-1 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fabiocat93 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

batwood-1 commented Dec 9, 2024 •

edited

Loading