Redirection and Utterance Likelihood

The Redirection transformer measures the extent to which utterances redirect the flow of the conversation, as described in this paper. The redirection effect of an utterance is determined by comparing the likelihood of its reply given the immediate conversation context vs. a reference context representing the previous direction of the conversation.

The UtteranceLikelihood transformer is a more generalized module that just implements log-likelihoods of utterances given a defined conversation context.

Example usage: redirection in supreme court oral arguments

class convokit.redirection.redirection.Redirection(likelihood_model, previous_context_selector=<function default_previous_context_selector>, future_context_selector=<function default_future_context_selector>, redirection_attribute_name='redirection')

ConvoKit transformer to compute redirection scores, derived from utterance probabilities from likelihood_model. The contexts used to compute redirection can be defined using previous_context_selector and future_context_selector, which are by default the immediate previous and future contexts from different speaker roles.

Parameters

  • likelihood_model – Likelihood model to compute utterance likelihoods

  • previous_context_selector – Computes tuple of actual, reference contexts used for redirection

  • future_context_selector – Computes future contexts used for redirection

  • redirection_attribute_name – Name of meta-data attribute to save redirection scores

fit(corpus, train_selector=<function Redirection.<lambda>>, val_selector=<function Redirection.<lambda>>)

Fits the redirection transformer to the corpus by generating the training and validation data and fine-tuning the likelihood model.

Parameters

  • corpus – Corpus to fit transformer

  • train_selector – Selector for train conversations

  • val_selector – Selector for val conversations

fit_transform(train_selector=<function Redirection.<lambda>>, val_selector=<function Redirection.<lambda>>, test_selector=<function Redirection.<lambda>>, verbosity=10)

Fit and transform the model.

Parameters

  • corpus – Corpus to transform

  • train_selector – Selector for train data

  • val_selector – Selector for val data

  • test_selector – Selector for test data

  • verbosity – Verbosity for update messages

Returns

Corpus where test data is labeled with redirection scores

summarize(corpus, top_sample_size=10, bottom_sample_size=10)

Summarizes redirection transformer with high and low redirecting utterances.

Parameters

  • corpus – Corpus to analyze

  • top_sample_size – Number of utterances to print for high redirection

  • bottom_sample_size – Number of utterances to print for low redirection

transform(corpus, selector=<function Redirection.<lambda>>, verbosity=5)

Populates the corpus test data with redirection scores, by computing previous and future contexts, determining actual and reference likelihoods, and calculating redirection scores.

Parameters

  • corpus – Corpus to transform

  • selector – Selector for test data

  • verbosity – Verbosity for update messages

Returns

Corpus where test data is labeled with redirection scores

class convokit.redirection.likelihoodModel.LikelihoodModel

Abstract class representing a model to compute utterance likelihoods based on provided context. Different models (Gemma, Llama, Mistral, etc.) can be supported by inheriting from this base class.

abstract fit(train_data, val_data)

Fine-tunes the likelihood model on the provided train_data and validates on val_data.

Parameters

  • train_data – Data to fine-tune model

  • val_data – Data to validate model

property name

Name of the likelihood model.

abstract transform(test_data)

Computes the utterance likelihoods for the provided test_data.

Parameters

test_data – Data to compute likelihoods over

Returns

Likelihoods of the test_data

class convokit.utteranceLikelihood.utteranceLikelihood.UtteranceLikelihood(likelihood_model, previous_context_selector=None, future_context_selector=None, likelihood_attribute_name='utterance_likelihood')

ConvoKit transformer to compute utterance log-likelihoods derived from likelihood_model. The contexts used to compute the likelihoods can be defined using previous_context_selector and future_context_selector, which are by default the immediate previous and current contexts from different speaker roles.

Parameters

  • likelihood_model – Likelihood model to compute utterance log-likelihoods

  • previous_context_selector – Computes previous contexts

  • future_context_selector – Computes future contexts

  • likelihood_attribute_name – Name of meta-data attribute to save likelihoods

fit(corpus, train_selector=<function UtteranceLikelihood.<lambda>>, val_selector=<function UtteranceLikelihood.<lambda>>)

Fits the UtteranceLikelihood transformer to the corpus by generating the training and validation data and fine-tuning the likelihood model.

Parameters

  • corpus – Corpus to fit transformer

  • train_selector – Selector for train conversations

  • val_selector – Selector for val conversations

fit_transform(train_selector=<function UtteranceLikelihood.<lambda>>, val_selector=<function UtteranceLikelihood.<lambda>>, test_selector=<function UtteranceLikelihood.<lambda>>, verbosity=10)

Fit and transform the model.

Parameters

  • corpus – Corpus to transform

  • train_selector – Selector for train data

  • val_selector – Selector for val data

  • test_selector – Selector for test data

  • verbosity – Verbosity for update messages

Returns

Corpus where test data is labeled with utterance likelihoods

summarize(corpus, top_sample_size=10, bottom_sample_size=10)

Summarizes UtteranceLikelihood transformer using utterances with high and low probabilities.

Parameters

  • corpus – Corpus to analyze

  • top_sample_size – Number of utterances to print for high probabilities

  • bottom_sample_size – Number of utterances to print for low probabilities

transform(corpus, selector=<function UtteranceLikelihood.<lambda>>, verbosity=5)

Populates the corpus test data with utterance likelihoods, by first computing previous and future contexts.

Parameters

  • corpus – Corpus to transform

  • selector – Selector for test data

  • verbosity – Verbosity for update messages

Returns

Corpus where test data is labeled with utterance likelihoods