Overview Xtract uses command-line arguments to convert XML data into a tab-delimited table. -pattern places the data from individual records into separate rows. -element extracts values from specified fields into separate columns. -group, -block, and -subset limit element exploration to selected XML subregions. Processing Flags -strict Remove HTML and MathML tags -mixed Allow mixed content XML -self Allow detection of empty self-closing tags -accent Excise Unicode accents and diacritical marks -ascii Unicode to numeric HTML character entities -compress Compress runs of spaces -stops Retain stop words in selected phrases Data Source -input Read XML from file instead of stdin -transform File of substitutions for -translate -aliases Mappings file for -classify operation Exploration Argument Hierarchy -pattern Name of record within set -group Use of different argument -block names allows command-line -subset control of nested looping Path Navigation -path Explore by list of adjacent object names Exploration Constructs Object DateRevised Parent/Child Book/AuthorList Path MedlineCitation/Article/Journal/JournalIssue/PubDate Heterogeneous "PubmedArticleSet/*" Exhaustive "History/**" Nested "*/Taxon" Recursive "**/Gene-commentary" Current "*" Conditional Execution -if Element [@attribute] required -unless Skip if element matches -and All tests must pass -or Any passing test suffices -else Execute if conditional test failed -position [first|last|outer|inner|even|odd|all] String Constraints -equals String must match exactly -contains Substring must be present -mimics Containment test after converting punctuation to space -excludes Substring must be absent -includes Substring must match at word boundaries -is-within String must be present -starts-with Substring must be at beginning -ends-with Substring must be at end -is-not String must not match -is-before First string < second string -is-after First string > second string -consists-of String must only contain specified characters -matches Matches without commas, semicolons, or accents -resembles Requires all words, but in any order Object Constraints -is-equal-to Object values must match -differs-from Object values must differ Numeric Constraints -gt Greater than -ge Greater than or equal to -lt Less than -le Less than or equal to -eq Equal to -ne Not equal to Format Customization -ret Override line break between patterns -tab Replace tab character between fields -sep Separator between group members -pfx Prefix to print before group -sfx Suffix to print after group -rst Reset -sep through -elg -clr Clear queued tab separator -pfc Preface combines -clr and -pfx -deq Delete and replace queued tab separator -def Default placeholder for missing fields -lbl Insert arbitrary text XML Generation -set XML tag for entire set -rec XML tag for each record -wrp Wrap elements in XML object -enc Encase instance in XML object -plg Prologue to print before instance -elg Epilogue to print after instance -pkg Package subset in XML object -fwd Foreword to print before subset -awd Afterword to print after subset Tag and Attribute Construction -tag Start with "<" + object name -att Attribute key and literal string -atr Attribute key and element name -cls Close with ">" -slf Self-close with " />" -end End contents with "" FASTA Parsable Fields -bkt Wrap elements in bracketed fields Element Selection -element Print all items that match tag name -first Only print value of first item -last Only print value of last item -even Only print value of even items -odd Only print value of odd items -backward Print values in reverse order Variable Recording -NAME Record value in named variable --STATS Accumulate values into variable -element Constructs Tag Caption Group Initials,LastName Parent/Child MedlineCitation/PMID Unrestricted "PubDate/*" Attribute DescriptorName@MajorTopicYN Range MedlineDate[1:4] Substring "Title[phospholipase | rattlesnake]" Optional "Construct[can contain ^ vertical bar]" Object Count "#Author" Item Length "%Title" Element Depth "^PMID" Variable "&NAME" Special -element Operations Parent Index "+" Object Name "?" Object Value "~" XML Subtree "*" Children "$" Attributes "@" ASN.1 Record "." JSON Record "%" Numeric (Integer) Processing -num Count -len Length -sum Sum -acc Accumulator -min Minimum -max Maximum -inc Increment -dec Decrement -sub Difference -avg Arithmetic Mean -dev Deviation -med Median -mul Product -div Quotient -mod Remainder -geo Geometric Mean -hrm Harmonic Mean -rms Root Mean Square -sqt Square Root -lge Natural Logarithm -lg2 Logarithm Base 2 -log Logarithm Base 10 -bin Binary -oct Octal -hex Hexadecimal -bit Number of Bits Set Leading Zero Padding -pad 0-Pad to 8 digits Character Processing -encode XML-encode <, >, &, ", and ' characters -decode Base64-decode object embedded in XML -upper Convert text to upper-case -lower Convert text to lower-case -chain Change_spaces_to_underscores -title Capitalize initial letters of words -mirror Reverse order of letters -alpha Non-alphabetic characters to space -alnum Non-alphanumeric characters to space String Processing -basic Convert superscripts and subscripts -plain Remove embedded mixed-content markup tags -simple Normalize accented letters, spell Greek letters -author Multi-step author cleanup -journal Journal capitalization and punctuation cleanup -prose Text conversion to ASCII Text Processing -terms Partition text at spaces -words Split at punctuation marks -pairs Adjacent informative words -split Split using -with for delimiter -order Rearrange words in sorted order -reverse Reverse words in string -letters Separate individual letters -clauses Break at phrase separators -pentamers Sliding window of pentamers Citation Functions -year Extract first 4-digit year from string -month Match first month name, return as integer -date YYYY/MM/DD from -unit "PubDate" -date "*" -page Get digits (and letters) of first page number -auth Changed GenBank authors to Medline form -initials Parse initials from forename or given name Miscellaneous Functions -trim Remove extra spaces and leading zeros -wct Count number of -words in a string -doi Add https://doi.org/ prefix, URL encode -accession Allow indexing of full accession.version -numeric Only accept items that are entirely digits Value Transformation -translate Substitute values with -transform table -classify Substring word or phrase matches to -aliases table Regular Expression -replace Substitute text using regular expressions -reg Target expression -exp Replacement pattern Sequence Processing -fasta Split sequence into blocks of 70 uppercase letters Nucleotide Processing -revcomp Reverse complement nucleotide sequence -nucleic Subrange determines forward or revcomp -ncbi2na Expand ncbi2na to iupac -ncbi4na Expand ncbi4na to iupac (May need to truncate result to actual sequence length) -cds2prot Translate coding region using -gcode and (1-based) -frame Protein Processing -molwt Calculate molecular weight of peptide -molwt-m Molecular weight retaining initial methionine -molwt-f Keep initial M residue as formyl-methionine -pept Split amino acid runs at *, -, x, or X Sequence Coordinates -0-based Zero-Based -1-based One-Based -ucsc-based Half-Open Command Generator -insd Generate INSDSeq extraction commands -insdx Process -insd output table into XML -insd Argument Order Descriptors INSDSeq_sequence INSDSeq_definition INSDSeq_division Flags [complete|partial] Feature(s) CDS,mRNA Qualifiers INSDFeature_key "#INSDInterval" gene product feat_location sub_sequence Variation Processing -hgvs Convert sequence variation format to XML Frequency Table -histogram Collects data for sort-uniq-count on entire set of records Entrez Indexing -indexer Positional index using -wrp for field name Output Organization -head Print before everything else -tail Print after everything else -hd Print before each record -tl Print after each record Record Selection -select Select record subset by conditions -in File of identifiers to use for selection Record Rearrangement -sort Element to use as sort key -sort-fwd Alias of original -sort -sort-rev Sort records in reverse order -stable-sort Stable sort variants (including -fwd and -rev) Reformatting -format [copy|compact|flush|indent|expand] Validation -verify Report XML data integrity problems -test Check field for visible combining accent and invisible Unicode Summary -outline Display outline of XML structure -synopsis Display individual XML paths -contour Display XML paths to leaf nodes [delimiter] Full Exploration Command Precedence -pattern -path -division -group -branch -block -section -subset -unit -element Documentation -help Print this document -examples Examples of EDirect and xtract usage -unix Common Unix command arguments -version Print version number Notes String constraints use case-insensitive comparisons. Numeric constraints and selection arguments use integer values. -num and -len selections are synonyms for Object Count (#) and Item Length (%). -words, -pairs, -reverse, and -indexer convert to lower case. See transmute -help for data conversion and modification functions. Xtract Examples -pattern DocumentSummary -element Id -first Name Title -pattern "PubmedArticleSet/*" -block Author -sep " " -element Initials,LastName -pattern PubmedArticle -block MeshHeading -if "@MajorTopicYN" -equals Y -sep " / " -element DescriptorName,QualifierName -pattern GenomicInfoType -element ChrAccVer ChrStart ChrStop -pattern Taxon -block "*/Taxon" -unless Rank -equals "no rank" -tab "\n" -element Rank,ScientificName -pattern Entrezgene -block "**/Gene-commentary" -block INSDReference -position 2 -subset INSDInterval -position last -POS INSDInterval_to -element "&SEQ[&POS+1:]" -if Author -and Title -if "#Author" -lt 6 -and "%Title" -le 70 -if DateRevised/Year -gt 2005 -if ChrStop -lt ChrStart -if CommonName -contains mouse -if "&ABST" -starts-with "Transposable elements" -if MapLocation -element MapLocation -else -lbl "\-" -if inserted_sequence -differs-from deleted_sequence -min ChrStart,ChrStop -max ExonCount -inc position -element inserted_sequence -1-based ChrStart -tag Item -att type journal -cls -encode Source -end Item -tag Item -att type journal -atr name Source -slf -insd CDS gene product protein_id translation -insd complete mat_peptide "%peptide" product peptide -insd CDS INSDInterval_iscomp@value INSDInterval_from INSDInterval_to -insd source organism taxid -insd CDS gene product feat_intervals sub_sequence -pattern PubmedArticle -select PubDate/Year -eq 2015 -pattern PubmedArticle -select MedlineCitation/PMID -in file_of_pmids.txt -wrp PubmedArticleSet -pattern PubmedArticle -sort MedlineCitation/PMID -set PubmedArticleSet -pattern PubmedArticle -split-by-num 1000 -prefix "subset" -suffix "xml" -set PubmedArticleSet -pattern PubmedArticle -split-by-len 500000 -prefix "subset" -suffix "xml" -set PMCInfoSet -pattern PMCInfo -split-by-id PMCInfo/UID -prefix PMC -suffix xml -pattern PubmedBookArticle -path BookDocument.Book.AuthorList.Author -element LastName -pattern PubmedArticle -group MedlineCitation/Article/Journal/JournalIssue/PubDate -year "PubDate/*" -mixed -verify -find MedlineCitation/PMID -html -max 50 Transmute Examples transmute -j2x -set - -rec GeneRec transmute -t2x -set Set -rec Rec -skip 1 Code Name transmute -filter ExpXml decode content transmute -filter LocationHist remove object transmute -normalize pubmed transmute -head "" -tail "" -pattern "PubmedArticleSet/*" -format