Transformer documentation: se_tpb_speechgenerator

Transformer Purpose

Generates audio for a full-text dtbook file. The typical use is having a TTS system generate audio, but this is not a requirement. For example, a silent template audio file could be associated with each synch point in order to make an "empty" book ready for se_tpb_filesetcreator without the need for a possibly time consuming tts process.

Regardless the audio kind, attributes will be placed on elements representing synch points. Those attributes are smil:clipBegin, smil:clipEnd and smil:src with namespace URI http://www.w3.org/2001/SMIL20/.

Input Requirements

This transformer is written to work with a manuscript, that is a dtbook-2005-1 or dtbook-2005-2 document possibly enriched with elements and attributes from other namespaces. The input document must be "synch point normalized", see se_tpb_syncPointNormalizer for such transformation.

Some elements are supposed to be announced audible. Those elements must have attributes holding the say-before and say-after text strings. se_tpb_annonsator can be used to add those attributes to a dtbook document. Since those attribute names are configurable, make sure they match whatever se_tpb_annonsator uses.

Output

On success

Given the expected input the transformer outputs a manuscript, that is a dtbook-2005-1/2 document enriched with, among others, attributes indicating corresponding audio. Those attributes, smil:clipBegin, smil:clipEnd and smil:src, namespace URI http://www.w3.org/2001/SMIL20/, point out which elements should be represented by audio in the generated talking book. Output also includes the generated audio files referrenced by the smil-attributes.

sent-level synchronization should be used, although configurable. Other usage has not been tested.

On error

No specific recovery scheme. On error, this transformer will send a fatal message, then throw an exception and abort.

Configuration/Customization

Parameters (tdf)

inputFilename

required="true"
The input manuscript file.
Example: /home/books/manuscript.xml

outputDirectory

required="true"
Path to the output directory
Example: /home/books/audio

outputFilename

required="true"
The desired name of the output manuscript.
Example: /home/books/audio/speechgen-manuscript.xml

concurrentAudioMerge

required="false"
Whether the merge of the audio should be done concurrent to the speech generation or not. Due to license, some TTS systems spend most of their time sleeping just to avoid being too effective, Loquendo is an example of that. If that is the case, why not use the time doing something useful instead, like merging tiny audio clips? Parallel threads will be spawned to merge the audio.
Possible enum values:

• true
• false

Default: true

mp3Output

required="false"
Is mp3 the preferred audio output format? The default option is wav.
Possible enum values:

• true
• false

Default: false

sgConfigFilename

required="false"
Speech generator configuration file. See Speech Generator Configuration for details.
Example: /home/config/file.xml
Default: ${transformer_dir}/config/sgConfig.xml

ttsBuilderConfig

required="false"
The tts builder configuration file. See TTS Builder Configuration for details.
Example: /home/ttsbfiles/file.xml
Default: ${transformer_dir}/ttsbuilder.xml

ttsBuilderRNG

required="false"
Tests for the tts builder configuration file using relaxng with embedded schematron.
Example: /home/ttsbfiles/file.rng
Default: ${transformer_dir}/ttsbuilder-configtest.rng

Extended configurability

Speech Generator Configuration

The file pointed to by the tdf variable sgConfig provides the possibility to affect the processing of the document. Things like on which elements to synch, merge audio and so on, are configured there. A description of the possibilities follows together with a short example:

/sgConfig/absoluteSynch/item

The names of the the elements that should be synch points, no matter where they are.

/sgConfig/containsSynch/item

The name of the element for synch point level.

/sgConfig/announceAttributes/item

Elements of this type show which attributes contain announcements. Two elements of this kind is allowed, with id values before (which tells us about which attributes contains "say-before" content) and after (which tells us about which attributes contains "say-after" content). On those elements three attributes (plus id) must be placed:

• uri: the namespace uri of the announce-attributes.
• prefix: the namespace prefix.
• local: the attribute's local name.

The element body is empty.

/sgConfig/mergeAudio/item

Elements at which to divide the audio into different files. The values can be seen as the element-only xpath tail. level/hd rather than //level/hd, that is.

/sgConfig/silence

There is a possibility to add extra silence after and/or before certain events in the talking book. Silence is added at the end of a synch point, never at the beginning. In the current implementation, the duration of the desired silence is provided in milliseconds and extra silence can be added upon five different events:

• afterLast: After the last phrase in an audio clip. Typical usage would be when audio is merged at a heading, this ability would add silence just before the heading.
• afterFirst: After the first phrase in an audio clip. Typical usage would be just after a heading.
• beforeAnnouncement: Before an audible announcement.
• afterAnnouncement: After an audible announcement.
• afterRegularPhrase: After every regular phrase that's generated.

An example follows:

<?xml version="1.0" encoding="utf-8"?>
<sgConfig>

	<absoluteSynch>
		<item>pagenum</item>
		<item>noteref</item>
		<item>annoref</item>
		<item>linenum</item>
	</absoluteSynch>
	
	<containsSynch>
		<item>sent</item>
	</containsSynch>
	
	<announceAttributes>
		<item id="before" uri="http://www.daisy.org/ns/pipeline/annon" prefix="annon" local="before"/>
		<item id="after" uri="http://www.daisy.org/ns/pipeline/annon" prefix="annon" local="after"/>
	</announceAttributes>
	
	<mergeAudio>
		<item>h1</item>
		<item>h2</item>
		<item>h3</item>
		<item>h4</item>
		<item>h5</item>
		<item>h6</item>
		<item>level/hd</item>
	</mergeAudio>
	
	<silence>
		<afterLast>2000</afterLast>
		<afterFirst>800</afterFirst>
		<beforeAnnouncement>300</beforeAnnouncement>
		<afterAnnouncement>300</afterAnnouncement>
		<afterRegularPhrase>200</afterRegularPhrase>
	</silence>
	
</sgConfig>
	

TTS Builder Configuration

se_tpb_speechgenerator uses a simple factory to get hold of TTS implementations. The factory must be configured properly since it is not able to locate TTS systems on its own. The configuration consists of sections that are operating system specific. As subsections, there are language specific sections. Each language must contain no more than one TTS system. During runtime, the TTS Builder configuration file is validated using relaxng and schematron, but since a DTD is a compact way of showing a document's structure, here's one:

<!DOCTYPE ttsbuilder [
   <!ELEMENT ttsbuilder (os+)>
   <!ELEMENT os (property*, lang*)>
   <!ELEMENT property (EMPTY)>
   <!ELEMENT lang (tts)>
   <!ELEMENT tts (param+)>
   <!ELEMENT param (EMPTY)>
   
   <!ATTLIST property name  CDATA #REQUIRED>
   <!ATTLIST property match CDATA #REQUIRED>
   <!ATTLIST lang lang CDATA #REQUIRED>
   <!ATTLIST tts default (true) CDATA #IMPLIED>
   <!ATTLIST param name  CDATA #REQUIRED>
   <!ATTLIST param value CDATA #REQUIRED>
]>

Besides the rules expressible in a DTD, there are a few others, asserted using schematron:

• The length of the lang-attribute value must be 2. This is to follow the ISO-639 2-letter lower-case standard used in Java.
• lang-siblings don't have the same lang-attribute value.
• For each os, there is not more than one descendant tts which has got the attribute default="true" to be used as fallback.
• For each tts, there must not be two descendant params with the same value for attribute name.
• For each tts, there is a param with name attribute value=class

Configuration of a TTS mainly consists of parameters for a certain TTS wrapper, such as Java class name or path to binary TTS program. Each TTS system needs its own Java-wrapper, and hence their configuration can differ extensively. The wrapper communicate with the TTS system of your choice. The properties read from the TTS Builder Configuration are passed to the TTS Java wrapper constructor (if there is one taking a java.util.Map as parameter) and from there, it's up to the wrapper to decide what to do. This gives a developer great possibilities when it comes to creating a TTS wrapper and its configuration. If the Java wrapper extends se_tpb_speechgenerator.ExternalTTS, some functionality is available. By calling the void setParamMap(java.util.Map) the super class attempts to read the following parameters:

• generalRegexFilename - general "always use"-re:s. Absolute path to file.
• specificRegexFilename - book specific re:s. Absolute path to file.
• characterSubstitutionTables - a comma separated list of absolute file paths to character substitution tables. If this parameter is present, the program will look for the following two:
  • characterExcludeFromSubstitution - name of character set to exclude from substitution.
  • characterFallbackStates - what to do if no mapping is found, the following values are valid:

    fallbackToNonSpacingMarkRemovalTransliteration

    Determines whether a character substitution attempt should fallback to a transliteration to nonspacing mark removal (typically disaccentuation) attempt if a replacement text was not found in user provided substitution table(s).

    fallbackToLatinTransliteration

    Determines whether a character substitution attempt should fallback to a transliteration to Latin attempt if a replacement text was not found in user provided substitution table(s).

    fallbackToUCD

    Determines whether a character substitution attempt should fallback to names in the UCD table if a replacement text is not found in user provided substitution table(s).

• yearFilename. Absolute path to file.
• xsltFilename. Absolute path to file.

This will make calls to the following super class methods do something useful:

• String xsltFilter(Document) -Performs an xslt on the small DOM representing the synch point.
• String regexFilter(String) -Performs regex search-replace.
• String yearFilter(String) -Performs year specific regex search-replace, replacing numerals with text.
• String replaceUChars(String) and void replaceUChars(Node) (recursive) -Replaces characters, most likely configured (by you) to filter characters your TTS can't handle.

An example of the configuration follows:

<?xml version="1.0" encoding="UTF-8"?>

<!-- the Java class parameter must be supplied -->
<!-- ${transformer_dir} variable will be evaluated to the directory where se_tpb_speechgenerator resides. -->

<ttsbuilder>
	<!--******************************************************************************
	Windows
	*******************************************************************************-->
	<os>
		<!-- all properties must match java's System.getProperties()-properties.
			Standard regex match for an os to be usable in this program. -->
		<property name="os.name" match="[Ww]indows.*" />
		
		<lang lang="en">
			<!-- since xml:lang determines which tts to use when in 
			this program, provide only one tts per language! -->			
			
			<!-- this is configuration for one tts impl. the "default" attribute 
			should be set to true for one configuration for each os. -->
			<tts default="true">
				<!-- the Java class name -->
				<param name="class" value="se_tpb_speechgenerator.SAPIImpl"/>
				
				<!-- the binary SAPI-talking program used for tts conversion -->
				<param 
				    name="binary" 
				    value="${transformer_dir}/tts/SimpleCommandLineTTS/SimpleCommandLineTTS.exe"/>				
		
				<!-- an xml file containing simple search-replace regex rules. -->
				<param name="generalRegexFilename" value="${transformer_dir}/regex/richard.xml"/>
				
				<!-- book specific regexes, will be applied before "generalRegexFilename". -->
				<param 
				    name="specificRegexFilename" 
				    value="${transformer_dir}/regex/someBookSpecific-re.xml"/>
	
				<!-- xslt applied on each synchpoint -->
				<param name="xsltFilename" value="${transformer_dir}/xslt/transform.xsl"/>
	
				<!-- an xml file containing simple search-replace regex rules.
				    Those rules specifically replaces years in digits with text. -->
				<param name="yearFilename" value="${transformer_dir}/config/year_en.xml"/>
	
				<!-- SAPI specific parameter: The value will be used to embed the text in
					SAPI's xml-like way. This value will result in the following tags
					surrounding the input text: 
				<voice optional="Gender=Male"></voice>
				Where the starting point is <voice optional=""></voice>.
				More on SAPI xml codes: 
				http://msdn.microsoft.com/library/default.asp?url=/library/en-us/SAPI51sr/Whitepapers/WP_XML_TTS_Tutorial.asp
				-->
				<param name="sapiVoiceSelection" value="Gender=Male"/>
	
				<!-- An ability to filter characters and replace them with custom strings. -->
				<param 
				    name="characterSubstitutionTables" 
				    value="${transformer_dir}/character-translation-table.xml"/>
	
				<!-- The encoding of the character translation table. -->
				<param name="characterFallbackStates" value="fallbackToLatinTransliteration"/>		
			</tts>
		</lang>
		
		<lang lang="sv">
			<tts>
				<param name="class" value="se_tpb_speechgenerator.SAPIImpl"/>
				<param name="binary" value="${transformer_dir}/tts/SimpleCommandLineTTS/SimpleCommandLineTTS.exe"/> 
				<param name="generalRegexFilename" value="${transformer_dir}/regex/richard.xml"/>
				<param name="xsltFilename" value="${transformer_dir}/xslt/transform.xsl"/>
				<param name="yearFilename" value="${transformer_dir}/config/year_se.xml"/>
				<param name="sapiVoiceSelection" value="Language=41D"/>
				<param name="characterSubstitutionTables" value="${transformer_dir}/character-translation-table.xml"/>
				<param name="characterFallbackStates" value="fallbackToLatinTransliteration"/>
			</tts>
		</lang>
	</os>
	
	
	<!--******************************************************************************
	Linux
	*******************************************************************************-->
	<os>
		<property name="os.name" match="[Ll]inux.*" />
		
		<lang lang="en">
			<tts id="loquendo" default="true">
				<param name="class" value="se_tpb_speechgenerator.LoquendoImpl"/>
				<param name="binary" value="${transformer_dir}/../../../narratorLoquendo"/>
				<param name="generalRegexFilename" value="${transformer_dir}/regex/richard.xml"/>
				<param name="ttsProperties" value="${transformer_dir}/config/loquendo.xml"/>
				<param name="xsltFilename" value="${transformer_dir}/xslt/loquendo-en.xsl"/>
				<param name="yearFilename" value="${transformer_dir}/config/year_en.xml"/>
			</tts>
		</lang>
	</os>
</ttsbuilder>

TTS Java wrappers

If you need to use a TTS system other than SAPI, you must develop your own TTS Java wrapper. One way of doing that is to develop a class from scratch implementing se_tpb_speechgenerator.TTS. But the easiest way is to extend se_tpb_speechgenerator.ExternalTTS. The class is abstract, leaving three methods left to implement:

• long se_tpb_speechgenerator.ExternalTTS().sayImpl(Document syncPoint, File outputFile)
• long se_tpb_speechgenerator.ExternalTTS().sayImpl(String syncPoint, File outputFile)
• void se_tpb_speechgenerator.TTS.close()

The parameters configured in the TTS Builder Configuration will be passed to a constructor accepting a java.util.Map as a single parameter, otherwise they will be passed to the wrapper by a call to se_tpb_speechgenerator.ExternalTTS.setParamMap(java.util.Map). See the javadoc for more details. This lets you use the TTS system - and possible inter-process communication - of your choice. Once you have set up a proper TTS Builder Configuration your new TTS wrapper is ready to run.

Loquendo

At TPB we have been using a simple Java wrapper for the Loquendo TTS Linux version. Work has been made to make the TTS better for us. Some pre-processing rules have been developed using regexes, and those may come in handy for anyone using the SAPI version of the Loquendo TTS together with Narrator. Read more about what have been done: loquendo-preproc.html.

Further development

• Refactoring: Instead of letting se_tpb_speechgenerator figuring out which elements represent synch points by searching for certain element structures with text nodes, an attribute must be present on those elements, making the synch point search trivial. Identifying synch points should only be assigned to one transformer, and a possible candidate in the Narrator transformer chain would be se_tpb_syncPointNormalizer.
• RNG/Schematron test the Speech Generator Configuration file before running.
• Generate audio for non-empty dc:Creator and dc:Title. Since the fileset creator uses those elements in the absence of docAuthor and docTitle it would be nice to be able to supply audio as well.

Dependencies

May need to access some TTS system, which is not part of the Daisy Pipeline.

Author

Martin Blomberg, TPB.

Licensing

LGPL