{"ygPerms":{"resourceCapabilityList":[{"resourceType":"GROUP","capabilities":[{"name":"UPDATE"},{"name":"READ"},{"name":"WELCOME_MSG"},{"name":"PENDING_COUNT"},{"name":"ACTIVITY_LOGS"}]},{"resourceType":"PHOTO","capabilities":[{"name":"READ"},{"name":"READ"},{"name":"UPLOAD"},{"name":"UPDATE"},{"name":"DELETE"},{"name":"COPY"},{"name":"MOVE"},{"name":"APPROVE"},{"name":"UPLOADTEMP"},{"name":"GROUPCOVER"}]},{"resourceType":"FILE","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"UPDATE"},{"name":"DELETE"},{"name":"AUTO_SEND"}]},{"resourceType":"MEMBER","capabilities":[{"name":"READ"},{"name":"READ"},{"name":"EDIT_MODERATOR"},{"name":"DELETE"},{"name":"CREATE"},{"name":"UPDATE"},{"name":"BAN"}]},{"resourceType":"LINK","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"UPDATE"},{"name":"DELETE"},{"name":"MOVE_LINK"}]},{"resourceType":"CALENDAR","capabilities":[{"name":"READ"}]},{"resourceType":"DATABASE","capabilities":[{"name":"READ_DATA"},{"name":"CREATE"},{"name":"READ"}]},{"resourceType":"POLL","capabilities":[{"name":"READ"},{"name":"VOTE"},{"name":"UPDATE"},{"name":"CREATE"},{"name":"DELETE"},{"name":"CLOSE"}]},{"resourceType":"MESSAGE","capabilities":[{"name":"CREATE"},{"name":"READ"},{"name":"DELETE"}]},{"resourceType":"PENDING_MESSAGE","capabilities":[{"name":"READ"},{"name":"UPDATE"},{"name":"DELETE"}]},{"resourceType":"ATTACHMENTS","capabilities":[{"name":"READ"},{"name":"DELETE"}]},{"resourceType":"PHOTOMATIC_ALBUMS","capabilities":[{"name":"READ"},{"name":"UPLOAD"}]},{"resourceType":"MEMBERSHIP_TYPE","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"DELETE"},{"name":"UPDATE"}]},{"resourceType":"POST","capabilities":[{"name":"CREATE"},{"name":"UPDATE"},{"name":"READ"},{"name":"DELETE"}]},{"resourceType":"PIN","capabilities":[{"name":"CREATE"},{"name":"UPDATE"},{"name":"READ"},{"name":"DELETE"}]}],"modStatus":"MODERATOR","subStatus":"NORMAL","groupUrl":"groups.yahoo.com","intlCode":"us"},"comscore":"pageview_candidate","ygData":{"userId":6110318,"authorName":"Jean-Christophe Helary","from":"Jean-Christophe Helary &lt;helary@eskimo.com&gt;","profile":"jc_helary","replyTo":"LIST","senderId":"bfZBynhTAFrkAPalpXtwi_bF_UCYzz4xb7eosTYUWUYUFUTQEdHI-L5mF1YIvYKRKT14cw4KRoswPIneTfniq8aTLebO0V13uWT4eDYD7stD","spamInfo":{"isSpam":false,"reason":"0"},"subject":"Re: [OmT] Some glossary suggestions","postDate":"1149935868","msgId":5157,"canDelete":true,"contentTrasformed":false,"systemMessage":false,"headers":{"messageIdInHeader":"PDc4RDE3ODdBLTMwMEQtNDk1OS04NTJFLTQwOEUwQkMwREZENkBlc2tpbW8uY29tPg==","inReplyToHeader":"PGE0YjdkYjg5MDYwNjEwMDI0M3Q2NDc3NGUyMHFAbWFpbC5nbWFpbC5jb20+","referencesHeader":"PDQ0NzYxMjVELjYwMTAwMDRAYWJzYW1haWwuY28uemE+IDxhNGI3ZGI4OTA2MDYwOTA3Mzh4MmRhYzU5NWVwQG1haWwuZ21haWwuY29tPiA8NzAwNzJDREUtNzVDRS00NEJDLUJCOTItQ0I0RkZGRkVFRjQyQGVza2ltby5jb20+IDxhNGI3ZGI4OTA2MDYxMDAyNDN0NjQ3NzRlMjBxQG1haWwuZ21haWwuY29tPg=="},"prevInTopic":5156,"nextInTopic":5166,"prevInTime":5156,"nextInTime":5158,"topicId":5099,"numMessagesInTopic":14,"msgSnippet":"... Ok, I seen, so it clarifies the matches I described after. The problem is then the tokenization process itself. You remember that we applied Java","rawEmail":"Return-Path: &lt;helary@eskimo.com&gt;\r\nX-Sender: helary@eskimo.com\r\nX-Apparently-To: OmegaT@yahoogroups.com\r\nReceived: (qmail 37786 invoked from network); 10 Jun 2006 10:41:03 -0000\r\nReceived: from unknown (66.218.66.216)\n  by m30.grp.scd.yahoo.com with QMQP; 10 Jun 2006 10:41:03 -0000\r\nReceived: from unknown (HELO smtp10.tiki.ne.jp) (218.40.30.107)\n  by mta1.grp.scd.yahoo.com with SMTP; 10 Jun 2006 10:41:03 -0000\r\nReceived: from [192.168.11.2] (pl186.nas933.takamatsu.nttpc.ne.jp [210.136.182.186])\n\t(authenticated bits=0)\n\tby smtp10.tiki.ne.jp (8.12.11/8.12.11) with ESMTP id k5AAbmQ3027411\n\t(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO)\n\tfor &lt;OmegaT@yahoogroups.com&gt;; Sat, 10 Jun 2006 19:37:49 +0900 (JST)\n\t(envelope-from helary@eskimo.com)\r\nMime-Version: 1.0 (Apple Message framework v750)\r\nIn-Reply-To: &lt;a4b7db890606100243t64774e20q@mail.gmail.com&gt;\r\nReferences: &lt;4476125D.6010004@absamail.co.za&gt; &lt;a4b7db890606090738x2dac595ep@mail.gmail.com&gt; &lt;70072CDE-75CE-44BC-BB92-CB4FFFFEEF42@eskimo.com&gt; &lt;a4b7db890606100243t64774e20q@mail.gmail.com&gt;\r\nContent-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed\r\nMessage-Id: &lt;78D1787A-300D-4959-852E-408E0BC0DFD6@eskimo.com&gt;\r\nContent-Transfer-Encoding: 7bit\r\nDate: Sat, 10 Jun 2006 19:37:48 +0900\r\nTo: OmegaT@yahoogroups.com\r\nX-Mailer: Apple Mail (2.750)\r\nX-Originating-IP: 218.40.30.107\r\nX-eGroups-Msg-Info: 1:0:0:0\r\nFrom: Jean-Christophe Helary &lt;helary@eskimo.com&gt;\r\nSubject: Re: [OmT] Some glossary suggestions\r\nX-Yahoo-Group-Post: member; u=6110318; y=SFrEK-0pZrjh6e_90ea4pbSC6_8-wZ5diIFu2qlYtwsz9osrCA\r\nX-Yahoo-Profile: jc_helary\r\n\r\n&gt;&gt;  What would that do to have a:\n&gt;&gt;  if (glosTokens.containsAll(strTokens))\n&gt;&gt;      strEntry.addGlossaryEntry(glosEntry);\n&gt;\n&gt; The above if means:\n&gt; if the set of glossary tokens contains a set of all tokens in a\n&gt; segment -- unrealistic -- it should be vice versa as it is now.\n&gt; Why it doesn&#39;t work for Japanese, requires investigation.\n\nOk, I seen, so it clarifies the matches I described after.\n\nThe problem is then the tokenization process itself.\n\nYou remember that we applied Java tokenization to _all_ languages and  \nnot alphabet based languages only ? When was that ? RC7 ?\n\nBefore that TM matches as well as glossary matches worked on a single  \ncharacter base.\n\nI think what we need is tokenization + substring &quot;matching&quot; in  \ndefined tokens, for TM and for glossary items too.\n\nJC\n\n"}}