<?xml version="1.0" ?>
<!DOCTYPE book PUBLIC "-//KDE//DTD DocBook XML V4.2-Based Variant V1.1//EN" "dtd/kdex.dtd" [
  <!ENTITY kmyapplication "<application
>Sam</application
>">
  <!ENTITY kappname "&kmyapplication;">
  <!ENTITY package "kde-module">
  <!ENTITY % addindex "IGNORE">
  <!ENTITY % Dutch "INCLUDE">

]>
<book id="sam" lang="&language;">

<bookinfo>

&ged.vertaald;&Freek.de.Kruijf; 


<date
>2012-12-13</date>
<releaseinfo
>0.4</releaseinfo>

<abstract>
<para
>&kmyapplication; is een beheerder van een akoestisch model. </para>
</abstract>

<keywordset>
<keyword
>KDE</keyword>
<keyword
>kdeutils</keyword>
<keyword
>Kapp</keyword>
<keyword
>Simon</keyword>
<keyword
>herkenning</keyword>
<keyword
>spraakinvoer</keyword>
<keyword
>stem</keyword>
<keyword
>commando</keyword>
<keyword
>besturing</keyword>
<keyword
>model</keyword>
<keyword
>compilatie</keyword>
<keyword
>Sam</keyword>
<keyword
>toegankelijkheid</keyword>
</keywordset>

<legalnotice
>&FDLNotice;</legalnotice>

<copyright>
<year
>2009-2012</year>
<holder
>Peter Grasch</holder>
</copyright>

<authorgroup>
<author
><personname
> <firstname
>Peter</firstname
> <othername
>H.</othername
> <surname
>Grasch</surname
> </personname
> <email
>peter.grasch@bedahr.org</email
> </author>
</authorgroup>


<title
>Het handboek van &kmyapplication;</title>
</bookinfo>

<chapter id="introduction">
<title
>Inleiding</title>

<para
>&kmyapplication; is een hulpmiddel om akoestische modellen aan te maken en te testen. Het kan nieuwe spraakmodellen compileren, modellen gebruiken die door Simon zijn gemaakt en modellen produceren die later door Simon gebruikt kunnen worden.</para>
<para
>It is targeted towards people wanting more control over their acoustic model and provides much lower level access to the build process. &kmyapplication; is mainly geared towards speech professionals wanting to improve and / or test their acoustic model.</para>
<para
>For more information on the architecture of the Simon suite please see the <ulink url="help:/simon/overview.html#architecture"
>Simon manual</ulink
>.</para>

<sect1 id="background">
  <title
>Achtergrond</title>

  <para
>This section will provide a bit of background information on the compilation and testing process.</para>

<sect2 id="effective_testing">
  <title
>Effectief testen</title>
  <para
>Een van de hoofdfuncties van Sam is het testen van de gegenereerde akoestische modellen.</para>

  <para
>The basic testing procedure is to run recognition on samples where the transcription is already known and comparing the results. &kmyapplication; also takes the confidence score of the recognition into account to measure how robust the created system is.</para>

  <para
>Due to the way acoustic models are created, both the recognition accuracy and confidence will be highly skewed when the same samples are used both for training and testing. This is called "in corpus" testing (the samples used for testing are also in your training corpus).</para>

  <para
>While in corpus testing might tell you if the compilation process failed or produced sub par results it won't tell you the "real" recognition rate of the created model. Therefore it is recommended to do "out of corpus" testing: Use different samples for training and for testing.</para>
  <para
>For out of corpus testing simply split your prompts file in two parts: One used to compile the model and one to test the model. Of course the test set doesn't need to be very big to get a representative result.</para>
  <para
>If you don't have a lot of training data, you can also split the complete corpus in ten parts. Compile 10 models, each leaving out one part of the corpus. Then do then individual tests (always with the test set that was excluded during compilation) and average the results.</para>

</sect2>
</sect1>

</chapter>

<chapter id="using-kapp">
<title
>Using &kmyapplication;</title>

<para
>&kmyapplication; provides a graphical interface that is divided into five areas.</para>

<para>
<screenshot>
<screeninfo
>Hoofdvenster</screeninfo>
  <mediaobject>
    <imageobject>
      <imagedata fileref="main.png" format="PNG"/>
    </imageobject>
    <textobject>
      <phrase
>Hoofdvenster</phrase>
    </textobject>
  </mediaobject>
</screenshot>
</para>

<para
>The order of the tabs represents a complete Sam workflow from creating a model to testing it. </para>

<sect1 id="input_output">
  <title
>Input &amp; output files</title>

  <para
>Define what files to work on. You can save and load this configuration by using the <guibutton
>Save</guibutton
> and <guibutton
>Load</guibutton
> buttons respectively.</para>
<warning>
    <para
>The <guibutton
>Save</guibutton
> and <guibutton
>Load</guibutton
> menus <emphasis
>only</emphasis
> save the paths and options defined here. They don't save the associated files!</para>
</warning>

<para
>Here you also have the option to select between supported model compilation / test backends. Depending on your Simon installation, not all might be available.</para>

<para>
<screenshot>
<screeninfo
>Input &amp; output selection</screeninfo>
  <mediaobject>
    <imageobject>
      <imagedata fileref="input_output.png" format="PNG"/>
    </imageobject>
    <textobject>
      <phrase
>Input &amp; output selection</phrase>
    </textobject>
  </mediaobject>
</screenshot>
</para>

<para
>If you want to compile and / or test the Simon model, you can use the <guibutton
>Modify Simon's model</guibutton
> option to load the appropriate files. Both scenarios and prompts will be serialized as appropriate.</para>

<para
>When Simon compiles the model, it will automatically remove untrained words during serialization as well as adapting the prompts appropriately. This will also happen when selecting <guibutton
>Modify Simon's model</guibutton
>. If you do, however serialize them separately using <guibutton
>Serialize scenarios</guibutton
> and <guibutton
>Serialize prompts</guibutton
> this adaption will not be performed and you alone are responsible for the validity of the input files. If you provide file names directly this of course applies even more so.</para>

<para
>When selecting a static model as the used model type &kmyapplication; will still compile the language model, just like Simon would. It also copies the given base model input files to the output paths.</para>

<para
>&kmyapplication; allows the user to provide different types of prompts files for the creation and the testing of the model. For more information about how to use this feature effectively, please refer to the section on <link linkend="effective_testing"
>effective testing</link
>. </para>

<para>
<screenshot>
<screeninfo
>Testconfiguratie</screeninfo>
  <mediaobject>
    <imageobject>
      <imagedata fileref="sam_test_io.png" format="PNG"/>
    </imageobject>
    <textobject>
      <phrase
>Testconfiguratie</phrase>
    </textobject>
  </mediaobject>
</screenshot>
</para>

<para
>Users can set up an arbitrary number of test configurations to compare different model parameters. All configured tests will be run and all are listed in any reports that might be exported. Through the option of supplying different input files than the compiled model, reference models (baselines) can be included in the report. </para>

<para
>Since Simon 0.4, the primary file format for models is the newly introduced SBM format. While Sam will also create SBM models, the testing still requires low level input files. This may seem counter intuitive but presents certain advantages in practice: The Simon model compilation (and thus also the SAM model compilation) still builds the low level output files for both SPHINX and HTK model compilers before packaging them into SBMs so pointing SAM to these (temporary) files is not difficult and in fact the default when using <guibutton
>Modify Simon's model</guibutton
>. Because acoustic model compilation is typically an iterative process, the compiler typically produces multiple temporary versions of different stages of parameter estimation. By allowing low level input, testing and comparing these intermediate results is much easier. </para>
<para
>If you want to test a model you received as a SBM container, you can extract it with SAM through the <guibutton
>Extract Simon Model</guibutton
> option to gain access to the low level input files required for testing with &kmyapplication;. </para>

</sect1>

<sect1 id="conservative_training">
<title
>Conservatieve training</title>
<para
>If you have <ulink url="help:/simond/using-kapp.html#User_Configuration"
>configured Simond to keep recognition samples</ulink
>, Sam can import this data through the <guibutton
>Import recognition samples</guibutton
> option in the "Input &amp; output" section.</para>

<para>
<screenshot>
<screeninfo
>Conservatieve training</screeninfo>
  <mediaobject>
    <imageobject>
      <imagedata fileref="conservative_training.png" format="PNG"/>
    </imageobject>
    <textobject>
      <phrase
>Conservatieve training</phrase>
    </textobject>
  </mediaobject>
</screenshot>
</para>

<para
>The label of the prompts file will be built automatically from the most likely recognition result. You can limit the selection to only include samples that achieved a given confidence score.</para>

<para
>If the Simon setup you're importing the samples from uses a <ulink url="help:/simon/commands.html#filter_command_plugin"
>global filter to deactivate recognition</ulink
>, then you can provide Sam with the command that activates and deactivates this filter (pausing and resuming the recognition, respectively) to ignore any samples produced while the recognition was deactivated.</para>
</sect1>

<sect1 id="adapt_scenarios">
  <title
>Scenario's aanpassen</title>

  <para
>Simon stores the language model in scenarios. When you want to use them to compile the model, you first need to serialize them to files readable by SPHINX or Julius and the HTK.</para>

<para>
<screenshot>
<screeninfo
>Scenario's aanpassen</screeninfo>
  <mediaobject>
    <imageobject>
      <imagedata fileref="adapt_scenarios.png" format="PNG"/>
    </imageobject>
    <textobject>
      <phrase
>Scenario's aanpassen</phrase>
    </textobject>
  </mediaobject>
</screenshot>
</para>

  <para
>To do this, select <guibutton
>Serialize scenarios</guibutton
> button in the <link linkend="input_output"
>Input &amp; output tab</link
>. Simon's scenarios will also be serialized when using the <guibutton
>Modify Simon's model</guibutton
> options.</para>

  <para
>In this page you can find status information of the adaption and read detailed error messages if an error occurs.</para>
</sect1>

<sect1 id="build_model">
  <title
>Model aanmaken</title>

  <para
>Here you can build the model using the input and output files defined in the <link linkend="input_output"
>input &amp; output section</link
>.</para>

  <para
>The compilation process is identical to the one used by Simond. Unlike Simond, however, the full log of all called external programs, their output as well as progress information is shown regardless of whether the compilation was successful or not.</para>

<para>
<screenshot>
<screeninfo
>Model aanmaken</screeninfo>
  <mediaobject>
    <imageobject>
      <imagedata fileref="build.png" format="PNG"/>
    </imageobject>
    <textobject>
      <phrase
>Create model</phrase>
    </textobject>
  </mediaobject>
</screenshot>
</para>

  <para
>Using this verbose output it is much easier to find issues with the input data.</para>
</sect1>

<sect1 id="test">
  <title
>Model testen</title>
  <para
>Here you can test the speech model.</para>
  <para
>&kmyapplication; will test the model set by in the <guilabel
>Output files</guilabel
> section in the <link linkend="input_output"
>input &amp; output section</link
>.</para>

  <para
>During testing, the recognizer of the selected backend will run to recognize on the input files as set by the test prompts. Resulting word- and sentence errors will be counted as well as the overall robustness by analyzing confidence scores of the recognizer.</para>

  <para
>The main test screen will show all started tests and report when they complete. The tests themselves will run threaded (at the same time). The amount of parallelism best suited for your machine is determined automatically.</para>

<para>
<screenshot>
<screeninfo
>Model testen</screeninfo>
  <mediaobject>
    <imageobject>
      <imagedata fileref="test_model.png" format="PNG"/>
    </imageobject>
    <textobject>
      <phrase
>Model testen</phrase>
    </textobject>
  </mediaobject>
</screenshot>
</para>

<para
>As soon as all tests are finished, the <link linkend="test_results"
>test results</link
> will be displayed automatically.</para>


</sect1>

<sect1 id="test_results">
  <title
>Resultaten oefening</title>

  <para
>Na een succesvolle <link linkend="test"
>modeltest</link
> kunt u een gedetailleerd rapport van de nauwkeurigheid van het herkennen hier vinden.</para>

  <sect2 id="scoring">
    <title
>Waardering</title>

    <para
>To best reflect the recognition performance, &kmyapplication; uses multiple, ranked results for the tests.</para>

    <para
>A correctly recognized word or sentence will be scored with the confidence score the word achieved. If a word is recognized correctly but another, wrong, result is ranked higher, the accuracy of that recognition will be 0%.</para>

    <para
>The displayed overall recognition rate is the average of all confidence rates. The shown overall accuracy rating shows the average over all accuracy scores which represents the likelihood that given any word in any sentence, this word has been recognized correctly.</para>

  </sect2>

  <sect2 id="test_results_words">
    <title
>Woord</title>

    <para
>&kmyapplication; will list recognition accuracy for each word individually.</para>

    <para>
    <screenshot>
    <screeninfo
>Word results</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_words.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>Word results</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>

    <para
>If you have samples containing more than one word they will be segmented during recognition. Each word will be scored individually (although the different words of course still influence each other).</para>
  </sect2>

  <sect2 id="test_results_sentences">
    <title
>Zinnen</title>
    <para
>This section lists each prompt as "sentence".</para>

    <para
>Prompts that were recorded more than once are combined.</para>

    <para>
    <screenshot>
    <screeninfo
>Sentence results</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_sentences.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>Sentence results</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>
  </sect2>

  <sect2 id="test_results_files">
    <title
>Bestanden</title>
    <para
>In the files section you can see the recognition results for each file. Each file will list the <link linkend="scoring"
>10 most likely results</link
> in the details pane when you select it.</para>

    <para>
    <screenshot>
    <screeninfo
>File results</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_files.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>File results</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>

    <para
>When you identify problematic samples, you can re-record (or remove) them by selecting them and clicking on <guibutton
>Edit sample</guibutton
>.</para>

    <para
>You can sort the files by each column simply by clicking on the column header. This way it is very easy to find bad samples by sorting by recognition rate.</para>
  </sect2>

  <sect2 id="test_results_log">
    <title
>Log</title>
    <para
>Here you can review the full protocol of this subtest.</para>

    <para>
    <screenshot>
    <screeninfo
>Logboek van test</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_log.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>Logboek van test</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>
  </sect2>
</sect1>



<sect1 id="reports">
  <title
>Rapporten</title>

  <para
>&kmyapplication; can export reports about test results.</para>

  <para
>The export of a report can be started by selecting <guibutton
>Export test result</guibutton
> in the &kmyapplication; main window.</para>

  <sect2 id="templates">
    <title
>Sjablonen</title>
    <para
>Reports are created by filling in templates with information. Principally, these templates are either plain text files or LaTeX files.</para>

    <para
>Three templates of LaTeX templates, two German and one English one, are shipped with &kmyapplication; but adding new ones is easy. Just install new templates to one of the following locations depending on your operating system: <itemizedlist>
      <listitem
><para
>Linux: <filename
>~/.kde/share/apps/sam/reports/templates</filename
></para
></listitem>
      <listitem
><para
>Windows: <filename
>%appdata%\.kde\share\apps\sam\reports\templates</filename
></para
></listitem>
    </itemizedlist>
    </para>

    <para
>The existing templates can be used as references. They are to be found at the following path: <itemizedlist>
      <listitem
><para
>Linux: <filename
>/usr/share/apps/sam/reports/templates</filename
></para
></listitem>
      <listitem
><para
>Windows: <filename
>&lt;installation path&gt;\share\apps\sam\reports\templates</filename
></para
></listitem>
    </itemizedlist>
    </para>
  </sect2>


  <sect2 id="parameters">
    <title
>Providing meta-information</title>

    <para
>The reporting engine allows the input of meta information about the used input data as well as the test conditions, speakers, etc.</para>

    <para>
    <screenshot>
    <screeninfo
>Rapport: Algemene informatie</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_export_main.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>Rapport: Algemene informatie</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>

    <para
>This information can be entered directly through the graphical user interface. Depending on the used template, not all provided information might be used by the exported report.</para>

    <para>
    <screenshot>
    <screeninfo
>Rapport: Experimentele informatie</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_export_experiment.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>Rapport: Experimentele informatie</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>

    <para
>Information entered in the dialog will also be saved with your &kmyapplication; project file.</para>


    <para>
    <screenshot>
    <screeninfo
>Report: Test set</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_export_testset.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>Report: Test setinformation</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>

    <para
>Some meta information, like the amount of samples in your test- and training corpus and the system information can be either filled in by &kmyapplication; automatically or provided manually.</para>

  </sect2>

  <sect2 id="output">
    <title
>Uitvoer</title>

    <para
>Exported reports are stored as ZIP archives. In the case of LaTeX templates they may contain graphics in addition to the raw <filename
>.tex</filename
> file.</para>

    <para
>Below you can see the beginning of an exported full LaTeX report of a simple test of the Amarok scenario with the static Voxforge base model.</para>

    <para>
    <screenshot>
    <screeninfo
>Geëxporteerd rapport</screeninfo>
      <mediaobject>
        <imageobject>
          <imagedata fileref="test_result_export_pdf.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase
>Geëxporteerd rapport</phrase>
        </textobject>
      </mediaobject>
    </screenshot>
    </para>
  </sect2>
</sect1>




<sect1 id="command_line_args">
  <title
>Argumenten voor het shell-commando</title>

  <para
>&kmyapplication; provides command line arguments for the most common actions to automatize model creation and testing.</para>

  <para
>All actions provided as command line arguments will be taken immediately after &kmyapplication; has loaded. Options that build on one another (like compiling and testing) will be re-ordered automatically.</para>

<table frame='all'
><title
>Argumenten voor het shell-commando</title>
<tgroup cols='2' align='left' colsep='1' rowsep='1'>
<colspec colname='c1'/>
<colspec colname='c2'/>
<thead>
<row>
  <entry
>Argument</entry>
  <entry
>Beschrijving</entry>
</row>
</thead>
<tbody>
<row>
  <entry
><parameter
>-m &lt;type&gt;</parameter
></entry>
  <entry
>Selects the used backend; Provide either "sphinx" or "htk" to select the specified backend. Depending on your installation, "sphinx" might not be available.</entry>
</row>
<row>
  <entry
><parameter
>-s</parameter
></entry>
  <entry
>Retrieves all paths from Simon; (equivalent to <guibutton
>Modify Simon's model</guibutton
>)</entry>
</row>
<row>
  <entry
><parameter
>-c</parameter
></entry>
  <entry
>Starts the model compilation;</entry>
</row>
<row>
  <entry
><parameter
>-t</parameter
></entry>
  <entry
>Starts the model test;</entry>
</row>
<row>
  <entry
><parameter
>-e &lt;filename&gt;</parameter
></entry>
  <entry
>Exports a report to the given filename;</entry>
</row>
<row>
  <entry
><parameter
>-b</parameter
></entry>
  <entry
>Enables batch processing; &kmyapplication; will quit automatically after all arguments have been processed. The main window will not be shown.</entry>
</row>
<row>
  <entry
><parameter
>-l &lt;filename&gt;</parameter
></entry>
  <entry
>Export the complete build log to the given file;</entry>
</row>
<row>
  <entry
><parameter
>-w</parameter
></entry>
  <entry
>Automatically safe any opened &kmyapplication; project file when exiting the application;</entry>
</row>
<row>
  <entry
><parameter
>&lt;filename&gt;</parameter
></entry>
  <entry
>Load this &kmyapplication; project file during startup.</entry>
</row>
</tbody>
</tgroup>
</table>

</sect1>

</chapter>


<chapter id="faq">
<title
>Vragen en antwoorden</title>

<para
>In een poging om deze sectie altijd up-to-date te houden is deze beschikbaar op onze <ulink url="http://userbase.kde.org/Special:myLanguage/Simon/Troubleshooting_Guide"
>online wiki</ulink
>. </para>

</chapter>

<chapter id="credits">
<title
>Dankbetuigingen en licentie</title>

<para
>&kmyapplication; </para>
<para
>Programma copyright 2008-2010 Peter Grasch <email
>peter.grasch@bedahr.org</email
> </para>

<para
>Documentatie copyright &copy; 2009-2010 Peter Grasch <email
>peter.grasch@bedahr.org</email
> </para>

&meld.fouten;&vertaling.freek; 
&underFDL; &underGPL; </chapter>

<appendix id="installation">
<title
>Installatie</title>
<para
>Kijk op onze <ulink url="http://userbase.kde.org/Special:myLanguage/Simon/Installation"
>wiki</ulink
> voor instructies over installatie.</para>
</appendix>

&documentation.index;
</book>

<!--
Local Variables:
mode: xml
sgml-minimize-attributes:nil
sgml-general-insert-case:lower
sgml-indent-step:0
sgml-indent-data:nil
End:

vim:tabstop=2:shiftwidth=2:expandtab
kate: space-indent on; indent-width 2; tab-width 2; indent-mode none;
-->
