如何使用 Stanford CoreNLP Coreferences 模块通过最具代表性的提及来替换单词

Question

我正在尝试找出使用 Stanford Corenlp 的 Coreference 模块通过 "resolving"（用其替换单词）重写句子的方法。

想法是改写如下句子：

John drove to Judy’s house. He made her dinner.

进入

John drove to Judy’s house. John made Judy dinner.

这是我一直在胡闹的代码：

    private void doTest(String text){
    Annotation doc = new Annotation(text);
    pipeline.annotate(doc);


    Map<Integer, CorefChain> corefs = doc.get(CorefChainAnnotation.class);
    List<CoreMap> sentences = doc.get(CoreAnnotations.SentencesAnnotation.class);


    List<String> resolved = new ArrayList<String>();

    for (CoreMap sentence : sentences) {

        List<CoreLabel> tokens = sentence.get(CoreAnnotations.TokensAnnotation.class);

        for (CoreLabel token : tokens) {

            Integer corefClustId= token.get(CorefCoreAnnotations.CorefClusterIdAnnotation.class);
            System.out.println(token.word() +  " --> corefClusterID = " + corefClustId);


            CorefChain chain = corefs.get(corefClustId);
            System.out.println("matched chain = " + chain);


            if(chain==null){
                resolved.add(token.word());
            }else{

                int sentINdx = chain.getRepresentativeMention().sentNum -1;
                CoreMap corefSentence = sentences.get(sentINdx);
                List<CoreLabel> corefSentenceTokens = corefSentence.get(TokensAnnotation.class);

                String newwords = "";
                CorefMention reprMent = chain.getRepresentativeMention();
                System.out.println(reprMent);
                for(int i = reprMent.startIndex; i<reprMent.endIndex; i++){
                    CoreLabel matchedLabel = corefSentenceTokens.get(i-1); //resolved.add(tokens.get(i).word());
                    resolved.add(matchedLabel.word());

                    newwords+=matchedLabel.word()+" ";

                }




                System.out.println("converting " + token.word() + " to " + newwords);
            }


            System.out.println();
            System.out.println();
            System.out.println("-----------------------------------------------------------------");

        }

    }


    String resolvedStr ="";
    System.out.println();
    for (String str : resolved) {
        resolvedStr+=str+" ";
    }
    System.out.println(resolvedStr);


}

我目前能达到的最佳输出是

John drove to Judy 's 's Judy 's house . John made Judy 's her dinner .

这不是很精彩...

我很确定有一种更简单的方法可以实现我想要实现的目标。

理想情况下，我想将句子重新组织为 CoreLabel 列表，这样我就可以保留它们附加的其他数据。

感谢任何帮助。

Answer 1

挑战在于您需要确保令牌不是其代表性提及的一部分。例如，令牌 "Judy" 具有 "Judy 's" 作为其代表提及项，因此如果您将其替换为短语 "Judy 's"，您将得到双“'s”。

您可以通过比较它们的索引来检查令牌是否是其代表性提及的一部分。如果令牌的索引小于代表性提及的 startIndex 或大于代表性提及的 endIndex，则您应该只替换令牌。否则你只保留令牌。

代码的相关部分现在如下所示：

            if (token.index() < reprMent.startIndex || token.index() > reprMent.endIndex) {

                for (int i = reprMent.startIndex; i < reprMent.endIndex; i++) {
                    CoreLabel matchedLabel = corefSentenceTokens.get(i - 1); 
                    resolved.add(matchedLabel.word());

                    newwords += matchedLabel.word() + " ";

                }
            }

            else {
                resolved.add(token.word());

            }

此外，为了加快进程，您还可以将第一个 if 条件替换为：

if (chain==null || chain.getMentionsInTextualOrder().size() == 1)

毕竟，如果共指链的长度只有1，那么找一个有代表性的mention是没有用的。

Answer 2

private void doTest(String text){
    Properties props = new Properties();
    props.put("annotators", "tokenize, ssplit, pos, lemma, ner, parse, dcoref");
    StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
    Annotation doc = new Annotation(text);
    pipeline.annotate(doc);


    Map<Integer, CorefChain> corefs = doc.get(CorefChainAnnotation.class);
    List<CoreMap> sentences = doc.get(CoreAnnotations.SentencesAnnotation.class);


    List<String> resolved = new ArrayList<String>();

    for (CoreMap sentence : sentences) {

        List<CoreLabel> tokens = sentence.get(CoreAnnotations.TokensAnnotation.class);

        for (CoreLabel token : tokens) {

            Integer corefClustId= token.get(CorefCoreAnnotations.CorefClusterIdAnnotation.class);
            System.out.println(token.word() +  " --> corefClusterID = " + corefClustId);


            CorefChain chain = corefs.get(corefClustId);
            System.out.println("matched chain = " + chain);


            if(chain==null){
                resolved.add(token.word());
                System.out.println("Adding the same word "+token.word());
            }else{

                int sentINdx = chain.getRepresentativeMention().sentNum -1;
                System.out.println("sentINdx :"+sentINdx);
                CoreMap corefSentence = sentences.get(sentINdx);
                List<CoreLabel> corefSentenceTokens = corefSentence.get(TokensAnnotation.class);
                String newwords = "";
                CorefMention reprMent = chain.getRepresentativeMention();
                System.out.println("reprMent :"+reprMent);
                System.out.println("Token index "+token.index());
                System.out.println("Start index "+reprMent.startIndex);
                System.out.println("End Index "+reprMent.endIndex);
                if (token.index() <= reprMent.startIndex || token.index() >= reprMent.endIndex) {

                        for (int i = reprMent.startIndex; i < reprMent.endIndex; i++) {
                            CoreLabel matchedLabel = corefSentenceTokens.get(i - 1); 
                            resolved.add(matchedLabel.word().replace("'s", ""));
                            System.out.println("matchedLabel : "+matchedLabel.word());
                            newwords += matchedLabel.word() + " ";

                        }
                    }

                    else {
                        resolved.add(token.word());
                        System.out.println("token.word() : "+token.word());
                    }



                System.out.println("converting " + token.word() + " to " + newwords);
            }


            System.out.println();
            System.out.println();
            System.out.println("-----------------------------------------------------------------");

        }

    }


    String resolvedStr ="";
    System.out.println();
    for (String str : resolved) {
        resolvedStr+=str+" ";
    }
    System.out.println(resolvedStr);


}

给出了完美的答案。

John drove to Judy’s house. He made her dinner. -----> John drove to Judy 's house . John made Judy dinner . Tom is a smart boy. He know a lot of thing. -----> Tom is a smart Tom . Tom know a lot of thing .

如何使用 Stanford CoreNLP Coreferences 模块通过最具代表性的提及来替换单词

How to replace a word by its most representative mention using Stanford CoreNLP Coreferences module

java

nlp

stanford-nlp