1

So I have some text that I want to analyze as part of a statistical analysis. I'm specifically interested in the relationships of words. I'm looking for:

  • The most frequently used 'word pairs' eg. the words which based on their average frequency in the text are most regularly adjacent to each other.
  • The most distant words eg. frequently used words tend to have the most words between them on average.

While I relate to the first example as 'frequent word pairs' I'm trying to find out what to call those word combinations with the largest distance between them.

Update

To clarify as there have been a fair few comments: I'm not looking at the distance of any word pairs but those words where the usage is most frequently increased in a subset of text compared to the average corpus of words.

An example will be words with an increased word frequency in a specific subreddit on reddit.

Improve this question
5
  • 1
    I don't understand how to pick pairs "with the largest distance between them" -- if you have a sentence of two hundred words, why not just pick the first and last ones? If they occur together, they identify themselves. But everything else doesn't occur together -- how do you specify a pair?
    – John Lawler
    Commented Oct 30, 2018 at 20:49
  • Do you mean something like: "if the word 'foo' appears in the text, 'bar' is also likely to appear but will be further away from 'foo' than from any other commonly appearing words"?
    – Michael J.
    Commented Oct 30, 2018 at 21:13
  • What you say about distance between words reminds me of aspects of stylometry (the use of frequency of words and phrases to pin down the identity of the unknown or disputed author of a piece of writing). But your characterisation does sound counterintuitive, as others have said. Surely the greater the average distance between instances of individual words in a particular piece of writing, the less the overall number of uses. But also, it is not clear whether your study relates to spoken language in general, or to the English language in particular. What exactly do you mean?
    – Tuffy
    Commented Oct 30, 2018 at 21:47
  • I find that "prologue" and "epilogue" are often quite far apart.
    – Jim
    Commented Oct 31, 2018 at 1:25
  • Well I was wondering if there was a non-descriptive term for something like this or similar to it in concept. But I'll go with it. Descriptive is usually the best way to ensure it's generally understood.
    – Hans
    Commented Oct 31, 2018 at 19:58

2 Answers 2

Reset to default
1

I believe you are referring to pairs of words that appear together at a given frequency but instead of being adjacent these have some, or a specific number of words between them. This description leaves open several variations.

The term I would use for such words would be "Frequently grouped" or perhaps just "Grouped". This would be in contrast to what you already have as Frequently paired or Paired words.

Improve this answer
2
  • In my mind, the word "grouped" is too similar to the word "paired" and means "put together"/"considered a unit". That doesn't necessarily make me think of words that are distant. I think a little more context from the OP might help though.
    – Gwendolyn
    Commented Oct 30, 2018 at 23:20
  • 1
    I added some context in the comment above. Essentially I'll only be looking for words with an increased frequency in a certain topic, but I want to highlight the most unrelated pairs ie the increase in frequency is the inverse of the number of words used between these words. I was hoping there may be a better phrase than 'distant word pairs'.
    – Hans
    Commented Oct 30, 2018 at 23:48
1

As I understand the question, you have a very large set of pairs of words. It is very large because every word in the English language appears on one side or the other of the pairs. You wish to examine the frequency distribution of the number of words that separate the two words of each pair in some given text (I do hope, for your sanity, that the given text is not the whole corpus of the written language).

A 'word pair' is defined as two words that occur side by side, such as 'red hair'.

You want to know if there is a word that defines the opposite condition, that is words whose use in your sample of text is separated by the largest number of other words. I do not believe that there is such a word, but if I wanted to write about this subject I would first define "degrees of separation" as a probability distribution, for each pair of words, of the length of the gap between uses of that pair of words (as always, given a source of tranches of English text). I would need to define what I mean by degrees of separation if both words do not occur in that sample of English text. And I would then wish to examine the most extreme examples of that distribution.

It is possible, but I think unlikely, that you will find a single pair of words for which the distribution of the degrees of separation is at all points higher than any other pair of words. So, for example, "ostrich, overcoat" might on average be separated by 100 words, but "carbon, concrete" with a lower average degree of separation might have extreme examples of larger degrees of separation than "ostrich, overcoat ."

In short, you are seeking a single word for a condition that cannot be defined.

Improve this answer
1
  • I initially wanted to use reddit posts as a reference. The idea is that certain sub reddits have a higher frequency for certain words in the sub than they have in the general language. What I want to find are these words that have a significantly increased number of occurrences in the posts but at the same time don't appear near each other in the text - so have a low degree of relation.
    – Hans
    Commented Oct 30, 2018 at 23:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .