A'hUdZddlmZddlmZmZmZddlmZddl m Z ddl m Z ddl mZddlmZdd lmZmZdd lmZmZdd lmZdd lmZmZdd lmZddlmZddl m!Z!m"Z"m#Z#ddl$m%Z%ddl&m'Z'm(Z(m)Z)ddl*m+Z+m,Z,ddl-m.Z.m/Z/de0de1e2fdZ3ie'jhe#e'jje"e'jle!e'jnee'jpe,e'jree'jte+e'jve e'jxee'jzee'j|e/e'j~e.e'jee'jee'jee'jee'jee'jee'je%e'jeiZHe2e'eeIe(eIe eIe)ffeJd<ddde'deededee e)ffdZKddddee'deed ee2dede1ee e)ff d!ZLy)"z Loading datasets and evaluators.)Sequence)AnyOptionalUnion)BaseLanguageModel)Chain)TrajectoryEvalChain)PairwiseStringEvalChain)LabeledPairwiseStringEvalChain)CriteriaEvalChainLabeledCriteriaEvalChain)EmbeddingDistanceEvalChain"PairwiseEmbeddingDistanceEvalChain)ExactMatchStringEvaluator)JsonEqualityEvaluatorJsonValidityEvaluator)JsonEditDistanceEvaluator)JsonSchemaEvaluator)ContextQAEvalChainCotQAEvalChain QAEvalChain)RegexMatchStringEvaluator) EvaluatorType LLMEvalChainStringEvaluator)LabeledScoreStringEvalChainScoreStringEvalChain)PairwiseStringDistanceEvalChainStringDistanceEvalChainurireturnc ddlm}|d|}|dDcgc]}|c}S#t$r tdwxYwcc}w)aLoad a dataset from the `LangChainDatasets on HuggingFace `_. Args: uri: The uri of the dataset to load. Returns: A list of dictionaries, each representing a row in the dataset. **Prerequisites** .. code-block:: shell pip install datasets Examples -------- .. code-block:: python from langchain.evaluation import load_dataset ds = load_dataset("llm-math") r) load_datasetzXload_dataset requires the `datasets` package. Please install with `pip install datasets`zLangChainDatasets/train)datasetsr# ImportError)r r#datasetds c/home/kushmeetdev/Regenta/Chatbot/venv/lib/python3.12/site-packages/langchain/evaluation/loading.pyr#r#(s[, )/u56Gw' (!A ((   :    )s & >;_EVALUATOR_MAPN)llm evaluatorr+kwargsc |tvr,td|dttjt|}t |t r+ ddlm}|xs |ddd}|jd d |i|S|d i|S#t$r# ddl m}n#t$r tdwxYwYVwxYw#t$r}td |d |d }~wwxYw)a<Load the requested evaluation chain specified by a string. Parameters ---------- evaluator : EvaluatorType The type of evaluator to load. llm : BaseLanguageModel, optional The language model to use for evaluation, by default None **kwargs : Any Additional keyword arguments to pass to the evaluator. Returns ------- Chain The loaded evaluation chain. Examples -------- >>> from langchain.evaluation import load_evaluator, EvaluatorType >>> evaluator = load_evaluator(EvaluatorType.QA) zUnknown evaluator type: z Valid types are: r) ChatOpenAIzCould not import langchain_openai or fallback onto langchain_community. Please install langchain_openai or specify a language model explicitly. It's recommended to install langchain_openai AND specify a language model explicitly.zgpt-4*)modelseed temperaturezEvaluation with the z requires a language model to function. Failed to create the default 'gpt-4' model. Please manually provide an evaluation LLM or check your openai credentials.Nr+) r* ValueErrorlistkeys issubclassrlangchain_openair/r&&langchain_community.chat_models.openai Exceptionfrom_llm)r,r+r- evaluator_clsr/es r)load_evaluatorr?ds6&&yk!$~':':'<"=!> @  #9-M-.  7J'JC&}%%8#888&v&&3  #%?  &}o655    sNBB0 B- BB-B''B-*B0,B--B00 C9C  C)r+config evaluatorsr@c g}|D]9}|r|j|ini}|jt|fd|ii||;|S)aeLoad evaluators specified by a list of evaluator types. Parameters ---------- evaluators : Sequence[EvaluatorType] The list of evaluator types to load. llm : BaseLanguageModel, optional The language model to use for evaluation, if none is provided, a default ChatOpenAI gpt-4 model will be used. config : dict, optional A dictionary mapping evaluator types to additional keyword arguments, by default None **kwargs : Any Additional keyword arguments to pass to all evaluators. Returns ------- List[Chain] The loaded evaluators. Examples -------- >>> from langchain.evaluation import load_evaluators, EvaluatorType >>> evaluators = [EvaluatorType.QA, EvaluatorType.CRITERIA] >>> loaded_evaluators = load_evaluators(evaluators, criteria="helpfulness") r+)getappendr?)rAr+r@r-loadedr,_kwargss r)load_evaluatorsrGs\BFS /5&**Y+2 nYQCQ;Pf;P;PQRS M)M__doc__collections.abcrtypingrrrlangchain_core.language_modelsrlangchain.chains.baser1langchain.evaluation.agents.trajectory_eval_chainr langchain.evaluation.comparisonr *langchain.evaluation.comparison.eval_chainr (langchain.evaluation.criteria.eval_chainr r ,langchain.evaluation.embedding_distance.baserr%langchain.evaluation.exact_match.baser!langchain.evaluation.parsing.baserr*langchain.evaluation.parsing.json_distancer(langchain.evaluation.parsing.json_schemarlangchain.evaluation.qarrr%langchain.evaluation.regex_match.baserlangchain.evaluation.schemarrr'langchain.evaluation.scoring.eval_chainrr)langchain.evaluation.string_distance.baserrstrr6dictr#QACOT_QA CONTEXT_QAPAIRWISE_STRING SCORE_STRINGLABELED_PAIRWISE_STRINGLABELED_SCORE_STRINGAGENT_TRAJECTORYCRITERIALABELED_CRITERIASTRING_DISTANCEPAIRWISE_STRING_DISTANCEEMBEDDING_DISTANCEPAIRWISE_EMBEDDING_DISTANCE JSON_VALIDITY JSON_EQUALITYJSON_EDIT_DISTANCEJSON_SCHEMA_VALIDATION REGEX_MATCH EXACT_MATCHr*type__annotations__r?rGr4rHr)rts&$''<'QCULQHSSKTT )c)d4j)Hk.0!!#:   4  ))+I &&(C""$7-""$<!!#:**,K$$&@--/Q!6 !6!"$$&?#$((*=88)5l+T%[$:OOPP:(,>'>' # $>' >'  5/ !" >'H(,! %'% # $% TN %  %  %& '( %rH