{"ygPerms":{"resourceCapabilityList":[{"resourceType":"GROUP","capabilities":[{"name":"UPDATE"},{"name":"READ"},{"name":"WELCOME_MSG"},{"name":"PENDING_COUNT"},{"name":"ACTIVITY_LOGS"}]},{"resourceType":"PHOTO","capabilities":[{"name":"READ"},{"name":"READ"},{"name":"UPLOAD"},{"name":"UPDATE"},{"name":"DELETE"},{"name":"COPY"},{"name":"MOVE"},{"name":"APPROVE"},{"name":"UPLOADTEMP"},{"name":"GROUPCOVER"}]},{"resourceType":"FILE","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"UPDATE"},{"name":"DELETE"},{"name":"AUTO_SEND"}]},{"resourceType":"MEMBER","capabilities":[{"name":"READ"},{"name":"READ"},{"name":"EDIT_MODERATOR"},{"name":"DELETE"},{"name":"CREATE"},{"name":"UPDATE"},{"name":"BAN"}]},{"resourceType":"LINK","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"UPDATE"},{"name":"DELETE"},{"name":"MOVE_LINK"}]},{"resourceType":"CALENDAR","capabilities":[{"name":"READ"}]},{"resourceType":"DATABASE","capabilities":[{"name":"READ"},{"name":"READ_DATA"},{"name":"CREATE"}]},{"resourceType":"POLL","capabilities":[{"name":"READ"},{"name":"VOTE"},{"name":"UPDATE"},{"name":"CREATE"},{"name":"DELETE"},{"name":"CLOSE"}]},{"resourceType":"MESSAGE","capabilities":[{"name":"CREATE"},{"name":"READ"},{"name":"DELETE"}]},{"resourceType":"PENDING_MESSAGE","capabilities":[{"name":"READ"},{"name":"UPDATE"},{"name":"DELETE"}]},{"resourceType":"ATTACHMENTS","capabilities":[{"name":"READ"},{"name":"DELETE"}]},{"resourceType":"PHOTOMATIC_ALBUMS","capabilities":[{"name":"READ"},{"name":"UPLOAD"}]},{"resourceType":"MEMBERSHIP_TYPE","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"DELETE"},{"name":"UPDATE"}]},{"resourceType":"POST","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"DELETE"},{"name":"UPDATE"}]},{"resourceType":"PIN","capabilities":[{"name":"READ"},{"name":"CREATE"},{"name":"DELETE"},{"name":"UPDATE"}]}],"modStatus":"MODERATOR","subStatus":"NORMAL","groupUrl":"groups.yahoo.com","intlCode":"us"},"comscore":"pageview_candidate","ygData":{"userId":40439957,"authorName":"Didier Briel","from":"&quot;Didier Briel&quot; &lt;d.briel@free.fr&gt;","profile":"didier_briel","replyTo":"LIST","senderId":"5WA7drCxUsgdio6CDj-uxE8OQ3kEupFEPfVKPmGCelvbyKS82V8A0O7Ugqc_hXwntRIrii_Z71v2eoCcIEEg2i96la7vWP0","spamInfo":{"isSpam":false,"reason":"12"},"subject":"RE: [OmT] Findings on the Czech tokenizer","postDate":"1262858073","msgId":16296,"canDelete":true,"contentTrasformed":false,"systemMessage":false,"headers":{"messageIdInHeader":"PEFPRUhLQ0tPSU9HUEdDTkxERUtCT0VIQ0dHQUEuZC5icmllbEBmcmVlLmZyPg==","inReplyToHeader":"PGhpMWRobCt1dnNAZUdyb3Vwcy5jb20+"},"prevInTopic":16295,"nextInTopic":16297,"prevInTime":16295,"nextInTime":16297,"topicId":16143,"numMessagesInTopic":77,"msgSnippet":"... Milan Hubacek ... Tokenizer links. I have passed all the stuff to my java-literate colleagues, as I do not know how to properly embed it into the OmegaT","rawEmail":"Return-Path: &lt;d.briel@free.fr&gt;\r\nX-Sender: d.briel@free.fr\r\nX-Apparently-To: OmegaT@yahoogroups.com\r\nX-Received: (qmail 27985 invoked from network); 7 Jan 2010 09:59:44 -0000\r\nX-Received: from unknown (66.196.94.106)\n  by m14.grp.re1.yahoo.com with QMQP; 7 Jan 2010 09:59:44 -0000\r\nX-Received: from unknown (HELO smtp1-g21.free.fr) (212.27.42.1)\n  by mta2.grp.re1.yahoo.com with SMTP; 7 Jan 2010 09:59:44 -0000\r\nX-Received: from smtp1-g21.free.fr (localhost [127.0.0.1])\n\tby smtp1-g21.free.fr (Postfix) with ESMTP id A29119401C0\n\tfor &lt;OmegaT@yahoogroups.com&gt;; Thu,  7 Jan 2010 10:54:37 +0100 (CET)\r\nX-Received: from Acer (ril69-2-82-235-41-242.fbx.proxad.net [82.235.41.242])\n\tby smtp1-g21.free.fr (Postfix) with SMTP id B5CF99401D3\n\tfor &lt;OmegaT@yahoogroups.com&gt;; Thu,  7 Jan 2010 10:54:34 +0100 (CET)\r\nTo: &lt;OmegaT@yahoogroups.com&gt;\r\nDate: Thu, 7 Jan 2010 10:54:33 +0100\r\nMessage-ID: &lt;AOEHKCKOIOGPGCNLDEKBOEHCGGAA.d.briel@free.fr&gt;\r\nMIME-Version: 1.0\r\nContent-Type: text/plain;\n\tcharset=&quot;iso-8859-1&quot;\r\nContent-Transfer-Encoding: 7bit\r\nX-Priority: 3 (Normal)\r\nX-MSMail-Priority: Normal\r\nX-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0)\r\nIn-Reply-To: &lt;hi1dhl+uvs@eGroups.com&gt;\r\nImportance: Normal\r\nX-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579\r\nX-Originating-IP: 212.27.42.1\r\nX-eGroups-Msg-Info: 1:12:0:0:0\r\nFrom: &quot;Didier Briel&quot; &lt;d.briel@free.fr&gt;\r\nSubject: RE: [OmT] Findings on the Czech tokenizer\r\nX-Yahoo-Group-Post: member; u=40439957; y=tgjYxZBKvJR7flKlujEXZCMLgDxYn1O8Mrojk_nOviPQE40oEmjX\r\nX-Yahoo-Profile: didier_briel\r\n\r\n-----Original Message-----\n&gt;From: OmegaT@yahoogroups.com [mailto:OmegaT@yahoogroups.com]On Behalf Of\nMilan Hubacek\n&gt;Sent: Wednesday, January 06, 2010 8:19 AM\n&gt;To: OmegaT@yahoogroups.com\n&gt;Subject: Re: [OmT] Findings on the Czech tokenizer\n\n&gt;Thanks for your comment, Didier, and also for the previously given CZech\nTokenizer links. I have passed all the stuff to my java-literate colleagues,\nas I do not know how to properly embed it into the OmegaT tokenizer. (Are\nthere any instructions available on the net?)\n\nNo, there are no instructions, because it&#39;s pure development, not simply\n&quot;configuration&quot;.\n\nAlex has kindly assessed the work that would have to be done to use these\nnon-Lucene tokenizers.\n\nContrary to Lucene tokenizers, they only contains stemmers for a single word\n(and stop words), they do not split sentences by tokens (i.e., words).\nSo one would have to:\nSplit sentence using Lucene&#39;s classes.\nRemove stop words.\nStem the remaining words.\n\nAccording to Alex, this is doable in a reasonable time, i.e. days, not\nweeks.\n\nDidier\n\n\n\n"}}