mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-22 21:23:02 +01:00
* iOS keyboard: a failure from Hex shows for seconds, not forever, and the Hex key retries it X testers on build 56 and earlier: "The keyboard keeps either bringing me back to the app or telling me another app is using the microphone" (@itsjling) and "The another app is using audio message seems inaccurate cause it's showing for me all the time although it's working fine" (@RedaSjakur). Cause, reproduced on build 56 (2ef08aa4) in an iOS 27.2 simulator on the Mac Studio (no microphone input): Hex's dictation failed ("Hex couldn't start the microphone. Try again."), Hex was then closed, and a minute later the Hex keyboard's strip in Settings' search field still said so. Hex writes its snapshot only while it runs, so a failed snapshot (heartbeat 0, no time) stayed in the App Group, and KeyboardViewModel read any .failed snapshot as failed whatever its age. The code path is unchanged on origin/dev. With a warm session up it was worse: Hex stayed in .failed, consumeKeyboardCommand ignored every start unless the phase was .ready, and the full keyboard's Hex key (isWarm = Full Access && warm available) kept sending starts to it, so nothing recorded until Hex was opened. The keyboard's own failure snapshot also reads as available for its 6 s, so a Hex key tap right after an in-keyboard failure went to a warm session that doesn't exist. Now: - KeyboardSnapshot.messageAt: when Hex wrote the message (DictationController stamps a failure when its phase becomes failed, and keyboardMessage when set; the keyboard's own failure stamps its snapshot). The keyboard drops a message, and reads a failure as offline, 8 s after it (messageShownFor), and at once for a snapshot with no time (an earlier build's). Whether a warm session is up and any result stay as they were. - KeyboardViewModel.dictatesThroughHex (Full Access, warm available, not failed), used by the full keyboard's Hex key and its touch-down capture: after a failure both keyboards dictate in the keyboard, as the Dictation bar already did. - Hex takes a keyboard start while failed when the failure was the microphone for a keyboard dictation with the model loaded (captureFailed): the next Hex key is its Try again. Model failures still wait for Hex to get ready again. Not changed: the 32 MB memory threshold and keyboardNeedsHex, the two ways the Hex key opens Hex. In the simulator the handoffs logged "speech model missing here" after the keyboard's own asset check failed ("anomaly.hex.mobile.keyboard is not subscribed to assistant.en_US") on a sim whose Hex had never installed a model, which is the intended path; a device's reason needs its probe log. Verified: new KeyboardBridgeTests.testAFailureHexWroteAWhileAgoIsOver and DictationControllerTests.testAWarmSessionsMicrophoneFailureIsRetriedFromTheKeyboard; KeyboardBridgeTests and DictationControllerTests pass (79 tests, 0 failures, iPhone 17 Pro iOS 27.2 simulator). In the simulator with this build, the same failure no longer shows in the keyboard a minute later. * iOS keyboard: the Hex key's audio session calls leave the main thread X tester @leeladhar (iPhone 16 Pro Max, build 56 or earlier): "the keyboard freezes while using apps sometimes ... the only way to close the keyboard is to close the app completely". Not reproduced in the iOS 27.2 simulator on build 56 (2ef08aa4) or origin/dev: 24 host visits (Messages, Contacts, Settings search) with typing, the Hex key and globe switches kept one keyboard process, no crash or hang report, footprint 29-40 MB with no growth. The device cause isn't confirmed. The likeliest one left on origin/dev is the last synchronous audio server round trips on the keyboard's main thread: every Hex key touch-down set the session's category, read and preferred its inputs (KeyboardLocalDictation.microphone(for:)), asked isOtherAudioPlaying, and read the Speech Recognition status for the probe log, all before the page could start. These wait on the audio server (and TCC) with no timeout; a build 56 crash report from TestFlight shows Hex's own main thread parked in such a call (setPreferredSampleRate waiting on SetBatchProperties). #2185 moved the one on the launch path for the same reason. While blocked, no key, Done or globe responds, and the keyboard can't dismiss. Now begin() runs those on the keyboard's audio session queue (after any earlier capture's release there) and starts the page when they return, unless the job was dropped, stopped or replaced meanwhile. Done before the page was asked to start ends quietly at once instead of leaving the page to start a capture nobody wants. Verified: builds clean; in an iPhone 17 Pro iOS 27.2 simulator (WebKit's mock audio device) the Hex key's touch-down capture still starts and is kept: "first audio +124 ms after touch-down (... getUserMedia +83 ...)", 13.5 s recorded as 643,072 frames at 48 kHz (no gap), then transcription as before. * iOS keyboard: the capture page resumes an audio context iOS suspended mid-dictation TestFlight tester on X (iPhone 13 Pro Max, iOS 27, build 56 or earlier): "in progress recording dropped when I took a screenshot". What a screenshot does to an in-keyboard dictation, in an iPhone 17 Pro iOS 27.2 simulator on the Mac Studio (WebKit's mock audio device records there): - The host only going inactive doesn't stop it: with Control Center pulled down for 4 s mid-dictation the page kept sending audio (700,416 frames at 48 kHz over a 14.8 s recording, no gap). - A screenshot taken with Cmd-Shift-3 hid the keyboard and the probe log said "keyboard hidden while recording: finishing with what was said" at the shutter. The simulator can only take one through its hardware keyboard, which also turns on keyboard minimization, so it doesn't show which of the two hid it. On an iPhone, iOS 26 and 27 open the screenshot's full-screen preview over the app, and a keyboard that hides is torn down and its WebKit processes suspended, so a dictation can't keep recording through that. - On build 56 a keyboard hidden mid-recording lost the dictation: viewWillDisappear only dropped a touch-down capture, and the recording went with the view controller (reproduced in #2163). Since #2163 on origin/dev it stops, keeps what was said, and the text goes into the same field when the keyboard comes back within 30 s, else is offered as Paste dictation. The one way left for a capture to end early with the keyboard still up is the page's audio context: iOS may suspend or interrupt it for other audio and the page never resumed it, so no more audio came and the 6 s watchdog ended the dictation. The page now resumes a context that isn't running every 250 ms while it records, and stops trying when the capture stops. Not changed: a keyboard hidden by a screenshot's preview still ends the recording (with its text kept, as above). Verified: builds clean; in the simulator the touch-down capture still starts and records through Control Center with no gap (numbers above). * iOS: a spoken count reads "Test 1, 2, 3." on every speech model TestFlight (@kevin_miller, Nov 20, build 56 =2ef08aa4): "Test 1, 2, 3" came out "Test three." / "Test 3" from the keyboard's in-keyboard dictation and "test 123" in the app's Dictate box. What each speech model writes (macOS 27.2.1 on the Mac Studio, `say` clips, 4 voices): Apple's SpeechTranscriber (in-keyboard dictation) writes "Test one, 2, 3.", "Test, one, two, three.", "Testing 123.", "Counting one, two, three, four."; DictationTranscriber writes "Test 123", "Counting 1234"; Parakeet v2 through FluidAudio 0.15.5 (the app) writes "Test 1, 2, 3", "Testing 123" (no pauses), "1, 2, 3, testing". Responsible steps: - "Test three." / "Test 3": at build 56 Clean up what I say deleted every word over P(DELETE) 0.98 with no guard. The real Core ML model gives the count words but the last 0.97-0.999 ("Test one two three." -> "Test three.", "Test 1, 2, 3." -> "Test 1, 3."): it reads a count as a self-correction. CleanupGuard (#2132, after build 56) already keeps every count on origin/dev; this commit adds tests that pin it. - "test 123": Parakeet and DictationTranscriber write "123" themselves; no Hex step merges digits. Apple Intelligence's cleanup (same prompt, macOS 27.2 model) kept "Test 1, 2, 3." in 3 of 3 runs per input. - The paths still disagreed on origin/dev: the keyboard inserted "Test one, 2, 3." where the app inserted "Test 1, 2, 3.". Change: SpokenCleanupRules.spokenCounts, run by CleanupTagger.cleanIfEnabled right after spoken symbols (every style but Verbatim, English, Clean up what I say on or off, before the tagger, in the app and in-keyboard dictation), writes a count in digits: three numbers or more counting up from one, in words or digits, with only a comma or a space between them ("one, 2, 3", "one two three" -> "1, 2, 3"), and "123"... right after "test"/"testing" when it ends its clause ("Test 123" -> "Test 1, 2, 3"; "test 123 passed", "room 123", "Counting 1234" stay). Decision: the "123" split is limited to test/testing because a bare "123" is usually a real number. Format lists already never makes a list of a count (each item would be empty); tests prove it. README gains the Spoken counts paragraph. Verified on the iOS 27.2 simulator (Mac Studio), real Core ML tagger: TranscriptCleanupGuardTests 9, CleanupGuardTests 6, SpokenCleanupRulesTests 19, HinglishRomanizerTests 7, SpokenListsTests 10, InformalContractionTests 4, FrenchSpacingTests 3, CleanupTaggerTests 15, DictationControllerTests 50: 123 tests, 0 failures. New: testSpokenCountsBecomeDigits, testOtherNumbersStayAsSaid, testCountsAreNoList, testCountsKeepEveryNumberFromEverySpeechModel (also shows the guardless tagger's "Test three."), testACountReadsTheSameFromTheAppAndTheKeyboard (app with Parakeet's text vs keyboard with both Apple modules' text, every style, Lowercase).
550 lines
27 KiB
Swift
550 lines
27 KiB
Swift
import Foundation
|
|
|
|
/// Clean up what I say for Spanish, German, and Norwegian, which the English cleanup tagger
|
|
/// was never trained on. Run on Spanish, the tagger dropped negations ("Te digo que no puedo
|
|
/// ir" became "Te puedo ir"), so these languages get two word rules instead, each needing
|
|
/// the commas a pause leaves:
|
|
/// - the language's hesitation sounds go anywhere ("eh", "mmm"; `TranscriptFormatter`'s
|
|
/// list);
|
|
/// - a filler phrase set off by commas goes ("Es caro, o sea, no lo compro");
|
|
/// - a spoken correction ("a las 3, digo, a las 4") drops its marker and the words it
|
|
/// replaces, counted back from the nearest earlier word that matches the repair's first
|
|
/// word ("a"), or else is of its kind (a number, weekday, month, or today and tomorrow),
|
|
/// within the sentence and `maxReach` words. With no such word, the sentence stays as
|
|
/// said.
|
|
/// Like the tagger, it only removes words; `CleanupReconstruction` repairs the commas and
|
|
/// capitals next to a removal.
|
|
nonisolated enum SpokenCleanupRules {
|
|
/// How far back a correction looks for the words it replaces.
|
|
static let maxReach = 8
|
|
|
|
struct Rules {
|
|
/// Removed between commas, or at a sentence start before a comma.
|
|
let fillers: [[String]]
|
|
/// Start a correction when set off by commas ("3, digo, 4"), or by a sentence end
|
|
/// and a comma ("3. Digo, 4").
|
|
let markers: [[String]]
|
|
/// Markers clear enough to count without commas ("3 mejor dicho 4").
|
|
let plainMarkers: [[String]]
|
|
/// Words of one kind can replace each other ("martes, perdón, miércoles").
|
|
let kinds: [Set<String>]
|
|
/// Words that lose their comma when the filler after them goes ("Y, este, fuimos").
|
|
let noComma: Set<String>
|
|
|
|
/// The index in `kinds` of a bare word, with any word holding a digit a number.
|
|
func kind(of word: String) -> Int? {
|
|
if word.contains(where: \.isNumber) { return kinds.count }
|
|
return kinds.firstIndex { $0.contains(word) }
|
|
}
|
|
}
|
|
|
|
static func rules(for language: String?) -> Rules? {
|
|
let code = language.flatMap { Locale(identifier: $0).language.languageCode?.identifier }
|
|
switch code {
|
|
case "es": return spanish
|
|
case "de": return german
|
|
case "nb", "nn", "no": return norwegian
|
|
default: return nil
|
|
}
|
|
}
|
|
|
|
static func supports(_ language: String?) -> Bool {
|
|
rules(for: language) != nil
|
|
}
|
|
|
|
/// `text` with its fillers and spoken corrections removed, or `text` itself when
|
|
/// nothing goes or the language has no rules.
|
|
static func clean(_ text: String, language: String?) -> String {
|
|
guard let rules = rules(for: language) else { return text }
|
|
let words = CleanupTagger.words(in: text)
|
|
var cleaned = removingFillers(words, rules: rules, language: language)
|
|
cleaned = resolvingCorrections(cleaned, rules: rules)
|
|
return cleaned == words ? text : cleaned.joined(separator: " ")
|
|
}
|
|
|
|
// MARK: Fillers
|
|
|
|
private static func removingFillers(_ input: [String], rules: Rules, language: String?) -> [String] {
|
|
var words = input
|
|
var deletes = Array(repeating: false, count: words.count)
|
|
let hesitation = TranscriptFormatter.fillerPattern(for: language).flatMap {
|
|
try? NSRegularExpression(pattern: "^(?:\($0))$", options: .caseInsensitive)
|
|
}
|
|
var index = 0
|
|
while index < words.count {
|
|
if let hesitation, isHesitation(words[index], hesitation) {
|
|
deletes[index] = true
|
|
index += 1
|
|
continue
|
|
}
|
|
guard let phrase = rules.fillers.first(where: { matches($0, words, at: index) }) else {
|
|
index += 1
|
|
continue
|
|
}
|
|
let end = index + phrase.count - 1
|
|
let opens = index == 0 || isPause(words[index - 1]) || endsSentence(words[index - 1])
|
|
guard opens, words[end].hasSuffix(","), end + 1 < words.count, !hasOpeningMark(words[index]) else {
|
|
index += 1
|
|
continue
|
|
}
|
|
if index > 0, words[index - 1].hasSuffix(","), rules.noComma.contains(bare(words[index - 1])) {
|
|
words[index - 1].removeLast()
|
|
}
|
|
for position in index...end {
|
|
deletes[position] = true
|
|
}
|
|
index = end + 1
|
|
}
|
|
guard deletes.contains(true) else { return input }
|
|
return CleanupTagger.words(in: CleanupReconstruction.reconstruct(words, deletes: deletes))
|
|
}
|
|
|
|
/// A sound like "eh", unless it asks or exclaims ("¿eh?", "¡Eh, tú!").
|
|
private static func isHesitation(_ word: String, _ pattern: NSRegularExpression) -> Bool {
|
|
guard !hasOpeningMark(word), let last = word.last, last != "?", last != "!" else { return false }
|
|
let bare = bare(word)
|
|
return pattern.firstMatch(in: bare, range: NSRange(bare.startIndex..., in: bare)) != nil
|
|
}
|
|
|
|
// MARK: Corrections
|
|
|
|
private struct Correction {
|
|
/// The first replaced word, the marker's last word, and the repair's first word.
|
|
let start: Int
|
|
let markerEnd: Int
|
|
let repair: Int
|
|
}
|
|
|
|
private static func resolvingCorrections(_ input: [String], rules: Rules) -> [String] {
|
|
var words = input
|
|
var from = 0
|
|
// Each correction removes words, so this ends; the cap only guards against a mistake.
|
|
for _ in 0..<words.count {
|
|
guard let found = nextCorrection(in: words, from: from, rules: rules) else { break }
|
|
words[found.repair] = taking(words[found.start], into: words[found.repair])
|
|
var deletes = Array(repeating: false, count: words.count)
|
|
for position in found.start...found.markerEnd {
|
|
deletes[position] = true
|
|
}
|
|
words = CleanupTagger.words(in: CleanupReconstruction.reconstruct(words, deletes: deletes))
|
|
from = found.start + 1
|
|
}
|
|
return words
|
|
}
|
|
|
|
private static func nextCorrection(in words: [String], from: Int, rules: Rules) -> Correction? {
|
|
guard from < words.count else { return nil }
|
|
for markerStart in max(1, from)..<words.count {
|
|
for marker in rules.markers where matches(marker, words, at: markerStart) {
|
|
let markerEnd = markerStart + marker.count - 1
|
|
let repair = markerEnd + 1
|
|
guard repair < words.count, !hasOpeningMark(words[markerStart]) else { continue }
|
|
let before = words[markerStart - 1]
|
|
let setOff = (isPause(before) || endsSentence(before)) && words[markerEnd].hasSuffix(",")
|
|
let plain = rules.plainMarkers.contains(marker)
|
|
&& !endsSentence(before)
|
|
&& !isPause(words[markerEnd])
|
|
&& !endsSentence(words[markerEnd])
|
|
guard setOff || plain else { continue }
|
|
let first = bare(words[repair])
|
|
guard !first.isEmpty else { continue }
|
|
|
|
// The sentence the replaced words are in, at most `maxReach` words back.
|
|
var low = markerStart - 1
|
|
while low > 0, !endsSentence(words[low - 1]), markerStart - low < maxReach {
|
|
low -= 1
|
|
}
|
|
let candidates = (low..<markerStart).reversed()
|
|
var start = candidates.first { bare(words[$0]) == first }
|
|
if start == nil, let kind = rules.kind(of: first) {
|
|
start = candidates.first { rules.kind(of: bare(words[$0])) == kind }
|
|
}
|
|
guard let start else { continue }
|
|
return Correction(start: start, markerEnd: markerEnd, repair: repair)
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
/// The repair takes the place of the words it corrects: its first word takes the first
|
|
/// replaced word's capital and opening mark ("¿El martes, perdón, el miércoles?" becomes
|
|
/// "¿El miércoles?").
|
|
private static func taking(_ replaced: String, into repair: String) -> String {
|
|
let marks = String(replaced.prefix { openingMarks.contains($0) })
|
|
var result = repair
|
|
if !marks.isEmpty, !hasOpeningMark(result) {
|
|
result = marks + result
|
|
}
|
|
let source = replaced.dropFirst(marks.count).first
|
|
guard let source, source.isLetter,
|
|
let index = result.firstIndex(where: \.isLetter),
|
|
!result[result.index(after: index)...].contains(where: \.isUppercase) else {
|
|
return result
|
|
}
|
|
let letter = String(result[index])
|
|
let cased = source.isUppercase ? letter.uppercased() : letter.lowercased()
|
|
result.replaceSubrange(index...index, with: cased)
|
|
return result
|
|
}
|
|
|
|
// MARK: Words
|
|
|
|
private static let openingMarks: Set<Character> = ["¿", "¡", "\"", "“", "«", "(", "„"]
|
|
private static let closingMarks: Set<Character> = [",", ".", ";", ":", "!", "?", "…", "\"", "”", "»", ")"]
|
|
|
|
/// Lowercased, without opening marks or trailing punctuation.
|
|
static func bare(_ word: String) -> String {
|
|
var scalars = Substring(word)
|
|
while let first = scalars.first, openingMarks.contains(first) {
|
|
scalars = scalars.dropFirst()
|
|
}
|
|
while let last = scalars.last, closingMarks.contains(last) {
|
|
scalars = scalars.dropLast()
|
|
}
|
|
return scalars.lowercased()
|
|
}
|
|
|
|
private static func matches(_ phrase: [String], _ words: [String], at index: Int) -> Bool {
|
|
guard index + phrase.count <= words.count else { return false }
|
|
for (offset, part) in phrase.enumerated() {
|
|
let word = words[index + offset]
|
|
// Inside a phrase only a comma may follow a word ("no, espera,").
|
|
if offset < phrase.count - 1, let last = word.last, last != ",", closingMarks.contains(last) {
|
|
return false
|
|
}
|
|
if bare(word) != part { return false }
|
|
}
|
|
return true
|
|
}
|
|
|
|
private static func isPause(_ word: String) -> Bool {
|
|
guard let last = word.last else { return false }
|
|
return last == "," || last == ";" || last == ":"
|
|
}
|
|
|
|
private static func endsSentence(_ word: String) -> Bool {
|
|
var trimmed = Substring(word)
|
|
while let last = trimmed.last, "\"”»)".contains(last) {
|
|
trimmed = trimmed.dropLast()
|
|
}
|
|
guard let last = trimmed.last else { return false }
|
|
return ".!?…".contains(last)
|
|
}
|
|
|
|
private static func hasOpeningMark(_ word: String) -> Bool {
|
|
word.first.map(openingMarks.contains) ?? false
|
|
}
|
|
|
|
// MARK: Spoken symbols
|
|
|
|
/// English `text` with "@" for its spoken names, or `text` itself. "at the rate" (what
|
|
/// India calls @) becomes "@" only before an address's domain ("gmail dot com",
|
|
/// "gmail.com"), never in "at the rate of 5%" or "at the rate we're going"; "at sign"
|
|
/// always does. Before a domain, the word before (with any "dot"-joined parts) and the
|
|
/// domain join it as one lowercase address: "John at the rate Gmail dot com." becomes
|
|
/// "john@gmail.com.", "dot" becoming "." only there. Elsewhere "at sign" is a lone "@"
|
|
/// ("the @ key"). Words around it keep their spacing.
|
|
static func spokenSymbols(in text: String, language: String?) -> String {
|
|
guard SpeechLanguagePicker.isEnglish(language), text.range(of: "at", options: .caseInsensitive) != nil else {
|
|
return text
|
|
}
|
|
let tokens = splittingGluedDomains(symbolTokens(in: text), in: text)
|
|
var edits: [(range: Range<String.Index>, text: String)] = []
|
|
var index = 0
|
|
while index < tokens.count {
|
|
guard let marker = symbolMarker(tokens, at: index) else {
|
|
index += 1
|
|
continue
|
|
}
|
|
let next = marker.end + 1
|
|
if marker.ambiguous, next < tokens.count, bare(tokens[next].word) == "of" {
|
|
index = next
|
|
continue
|
|
}
|
|
if let domain = addressDomain(tokens, from: next) {
|
|
var start = index
|
|
var local: [String] = []
|
|
if let last = tokens[safe: index - 1], let part = localPart(last.word) {
|
|
local = [part]
|
|
start = index - 1
|
|
// "john dot smith at the rate…": the parts before join with dots too.
|
|
while start >= 2, tokens[start - 1].word.lowercased() == "dot",
|
|
let part = localPart(tokens[start - 2].word) {
|
|
local.insert(part, at: 0)
|
|
start -= 2
|
|
}
|
|
}
|
|
let address = (local.joined(separator: ".") + "@" + domain.labels.joined(separator: ".")).lowercased()
|
|
edits.append((tokens[start].range.lowerBound..<tokens[domain.end].range.upperBound, address + domain.trailing))
|
|
index = domain.end + 1
|
|
} else if !marker.ambiguous, bare(tokens[safe: next]?.word ?? "") != "language" {
|
|
// "good at sign language" is no symbol.
|
|
let last = tokens[marker.end].word
|
|
let trailing = String(last.reversed().prefix { closingMarks.contains($0) }.reversed())
|
|
edits.append((tokens[index].range.lowerBound..<tokens[marker.end].range.upperBound, "@" + trailing))
|
|
index = marker.end + 1
|
|
} else {
|
|
index = marker.end + 1
|
|
}
|
|
}
|
|
guard !edits.isEmpty else { return text }
|
|
var result = ""
|
|
var cursor = text.startIndex
|
|
for edit in edits {
|
|
result += text[cursor..<edit.range.lowerBound] + edit.text
|
|
cursor = edit.range.upperBound
|
|
}
|
|
return result + text[cursor...]
|
|
}
|
|
|
|
private struct SymbolToken {
|
|
let word: String
|
|
let range: Range<String.Index>
|
|
}
|
|
|
|
private static func symbolTokens(in text: String) -> [SymbolToken] {
|
|
var tokens: [SymbolToken] = []
|
|
var start: String.Index?
|
|
for index in text.indices {
|
|
if text[index].isWhitespace {
|
|
if let begin = start {
|
|
tokens.append(SymbolToken(word: String(text[begin..<index]), range: begin..<index))
|
|
start = nil
|
|
}
|
|
} else if start == nil {
|
|
start = index
|
|
}
|
|
}
|
|
if let begin = start {
|
|
tokens.append(SymbolToken(word: String(text[begin...]), range: begin..<text.endIndex))
|
|
}
|
|
return tokens
|
|
}
|
|
|
|
/// Apple's `SpeechTranscriber` glues a spoken domain to the word before it ("John at the
|
|
/// rategmail.com.", "Priya at signyahoo.com."): this splits "rate" or "sign" off again.
|
|
private static func splittingGluedDomains(_ tokens: [SymbolToken], in text: String) -> [SymbolToken] {
|
|
var result: [SymbolToken] = []
|
|
for (index, token) in tokens.enumerated() {
|
|
let lower = token.word.lowercased()
|
|
let before = tokens[safe: index - 1]?.word.lowercased()
|
|
let prefix: String? = if before == "the", tokens[safe: index - 2]?.word.lowercased() == "at",
|
|
lower.hasPrefix("rate") {
|
|
"rate"
|
|
} else if before == "at", lower.hasPrefix("sign") {
|
|
"sign"
|
|
} else {
|
|
nil
|
|
}
|
|
guard let prefix, isDomain(String(token.word.dropFirst(prefix.count))) else {
|
|
result.append(token)
|
|
continue
|
|
}
|
|
let cut = text.index(token.range.lowerBound, offsetBy: prefix.count)
|
|
result.append(SymbolToken(word: String(token.word.prefix(prefix.count)), range: token.range.lowerBound..<cut))
|
|
result.append(SymbolToken(word: String(token.word.dropFirst(prefix.count)), range: cut..<token.range.upperBound))
|
|
}
|
|
return result
|
|
}
|
|
|
|
/// A written domain, maybe with punctuation after it: "gmail.com.", "co.in".
|
|
private static func isDomain(_ word: String) -> Bool {
|
|
let core = String(word.reversed().drop { closingMarks.contains($0) }.reversed())
|
|
let labels = core.split(separator: ".", omittingEmptySubsequences: false).map(String.init)
|
|
guard labels.count >= 2, let top = labels.last, top.count >= 2, top.allSatisfy(\.isLetter) else { return false }
|
|
return labels.allSatisfy(isDomainLabel)
|
|
}
|
|
|
|
/// "at the rate" (ambiguous: also plain English) or "at sign" / "at-sign" at `index`; the
|
|
/// words inside may not carry punctuation, the last may ("at sign.").
|
|
private static func symbolMarker(_ tokens: [SymbolToken], at index: Int) -> (end: Int, ambiguous: Bool)? {
|
|
func word(_ offset: Int) -> String? { tokens[safe: index + offset]?.word.lowercased() }
|
|
if bare(tokens[index].word) == "at-sign" { return (index, false) }
|
|
guard word(0) == "at" else { return nil }
|
|
if let sign = word(1), bare(sign) == "sign" { return (index + 1, false) }
|
|
if word(1) == "the", let rate = word(2), bare(rate) == "rate" { return (index + 2, true) }
|
|
return nil
|
|
}
|
|
|
|
/// A domain spoken from `index`: labels joined by "dot" or already by ".", at least two,
|
|
/// ending in a top-level domain of letters ("com", "co dot in"). Punctuation after the
|
|
/// last label (a sentence's full stop) is kept apart.
|
|
private static func addressDomain(
|
|
_ tokens: [SymbolToken],
|
|
from index: Int
|
|
) -> (labels: [String], end: Int, trailing: String)? {
|
|
var labels: [String] = []
|
|
var position = index
|
|
var trailing = ""
|
|
while let token = tokens[safe: position] {
|
|
let core = String(token.word.reversed().drop { closingMarks.contains($0) }.reversed())
|
|
let parts = core.split(separator: ".", omittingEmptySubsequences: false).map(String.init)
|
|
guard !core.isEmpty, parts.allSatisfy(isDomainLabel) else { return nil }
|
|
labels += parts
|
|
trailing = String(token.word.dropFirst(core.count))
|
|
guard trailing.isEmpty,
|
|
tokens[safe: position + 1]?.word.lowercased() == "dot",
|
|
let following = tokens[safe: position + 2],
|
|
isDomainLabel(String(following.word.reversed().drop { closingMarks.contains($0) }.reversed())
|
|
.split(separator: ".").first.map(String.init) ?? "") else { break }
|
|
position += 2
|
|
}
|
|
guard labels.count >= 2, let top = labels.last, top.count >= 2, top.allSatisfy(\.isLetter) else { return nil }
|
|
return (labels, position, trailing)
|
|
}
|
|
|
|
private static func isDomainLabel(_ label: String) -> Bool {
|
|
!label.isEmpty && label.count <= 63 && label.first != "-" && label.last != "-"
|
|
&& label.allSatisfy { ($0.isLetter || $0.isNumber || $0 == "-") && $0.isASCII }
|
|
}
|
|
|
|
/// The word before "@" when it can start an address: letters, digits, "." "_" "-" "+",
|
|
/// with no punctuation after it.
|
|
private static func localPart(_ word: String) -> String? {
|
|
guard !word.isEmpty, let last = word.last, last.isLetter || last.isNumber,
|
|
word.first.map({ $0.isLetter || $0.isNumber }) == true,
|
|
word.allSatisfy({ ($0.isLetter || $0.isNumber || "._-+".contains($0)) && $0.isASCII }) else { return nil }
|
|
return word
|
|
}
|
|
|
|
// MARK: Spoken counts
|
|
|
|
/// The English number words a count is made of.
|
|
private static let countWords: [String: Int] = [
|
|
"one": 1, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9, "ten": 10,
|
|
"eleven": 11, "twelve": 12, "thirteen": 13, "fourteen": 14, "fifteen": 15, "sixteen": 16, "seventeen": 17,
|
|
"eighteen": 18, "nineteen": 19, "twenty": 20,
|
|
]
|
|
|
|
/// English `text` with a spoken count in digits, or `text` itself, so a microphone test
|
|
/// reads the same from every speech model: for "test, one, two, three" Parakeet writes
|
|
/// "Test 1, 2, 3" and Apple's `SpeechTranscriber` "Test one, 2, 3." (macOS 27.2, `say`
|
|
/// voices), and the cleanup tagger is sure a count in words is a self-correction (P(DELETE)
|
|
/// over 0.98 on "one" and "two", which made "Test three." before `CleanupGuard`). A count is
|
|
/// three numbers or more counting up from one, in words or digits, with only a comma or a
|
|
/// space between them: "one two three" and "one, 2, 3" become "1, 2, 3". "123" (up to
|
|
/// "123456789") right after "test" or "testing" and ending its clause, which Parakeet and
|
|
/// `DictationTranscriber` write for a quick "testing one two three", becomes "1, 2, 3" too.
|
|
/// Every other number stays as said ("Chapter one, two and three", "at 1, 2, or 3", "room
|
|
/// 123", "test 123 passed").
|
|
static func spokenCounts(in text: String, language: String?) -> String {
|
|
guard SpeechLanguagePicker.isEnglish(language) else { return text }
|
|
let tokens = symbolTokens(in: text)
|
|
var edits: [(range: Range<String.Index>, text: String)] = []
|
|
var index = 0
|
|
while index < tokens.count {
|
|
if let digits = joinedCount(tokens, at: index) {
|
|
edits.append((tokens[index].range, digits))
|
|
index += 1
|
|
continue
|
|
}
|
|
var count: [(value: Int, trailing: String)] = []
|
|
while let token = tokens[safe: index + count.count], let number = countNumber(token.word),
|
|
number.value == count.count + 1 {
|
|
count.append(number)
|
|
// Only a comma or a space comes between two numbers of a count.
|
|
guard number.trailing.isEmpty || number.trailing == "," else { break }
|
|
}
|
|
// "zero, one, two, three" and "twenty one two three" are no count from one.
|
|
let previous = tokens[safe: index - 1].map { bare($0.word) } ?? ""
|
|
let followsNumber = countNumber(previous) != nil || previous == "zero"
|
|
|| (!previous.isEmpty && previous.allSatisfy(\.isNumber))
|
|
guard count.count >= 3, !followsNumber else {
|
|
index += 1
|
|
continue
|
|
}
|
|
let last = index + count.count - 1
|
|
let written = count.enumerated().map { offset, number in
|
|
"\(number.value)" + (offset == count.count - 1 ? number.trailing : ",")
|
|
}
|
|
edits.append((tokens[index].range.lowerBound..<tokens[last].range.upperBound, written.joined(separator: " ")))
|
|
index = last + 1
|
|
}
|
|
guard !edits.isEmpty else { return text }
|
|
var result = ""
|
|
var cursor = text.startIndex
|
|
for edit in edits {
|
|
result += text[cursor..<edit.range.lowerBound] + edit.text
|
|
cursor = edit.range.upperBound
|
|
}
|
|
return result + text[cursor...]
|
|
}
|
|
|
|
/// A count's number and the marks after it ("two," is 2 and ","): one of `countWords`, or 1
|
|
/// to 20 in digits.
|
|
private static func countNumber(_ word: String) -> (value: Int, trailing: String)? {
|
|
let core = String(word.reversed().drop { closingMarks.contains($0) }.reversed())
|
|
let trailing = String(word.dropFirst(core.count))
|
|
if let value = countWords[core.lowercased()] { return (value, trailing) }
|
|
guard core.count <= 2, core.first != "0", core.allSatisfy({ $0.isASCII && $0.isNumber }),
|
|
let value = Int(core) else { return nil }
|
|
return (value, trailing)
|
|
}
|
|
|
|
/// "123." after "test" or "testing" as "1, 2, 3.", when nothing but its marks or the end
|
|
/// follows it: "test 123 passed" keeps its number.
|
|
private static func joinedCount(_ tokens: [SymbolToken], at index: Int) -> String? {
|
|
guard let before = tokens[safe: index - 1], ["test", "testing"].contains(bare(before.word)),
|
|
!endsSentence(before.word) else { return nil }
|
|
let word = tokens[index].word
|
|
let core = String(word.reversed().drop { closingMarks.contains($0) }.reversed())
|
|
let trailing = word.dropFirst(core.count)
|
|
guard core.count >= 3, "123456789".hasPrefix(core), !trailing.isEmpty || index == tokens.count - 1 else {
|
|
return nil
|
|
}
|
|
return core.map(String.init).joined(separator: ", ") + trailing
|
|
}
|
|
|
|
// MARK: Languages
|
|
|
|
private static let spanish = Rules(
|
|
fillers: [["este"], ["o", "sea"]],
|
|
markers: [["digo"], ["perdón"], ["perdon"], ["no"], ["no", "espera"], ["no", "mejor"], ["mejor", "dicho"]],
|
|
plainMarkers: [["mejor", "dicho"], ["no", "mejor"]],
|
|
kinds: [
|
|
["dos", "tres", "cuatro", "cinco", "seis", "siete", "ocho", "nueve", "diez", "once", "doce"],
|
|
["lunes", "martes", "miércoles", "jueves", "viernes", "sábado", "domingo"],
|
|
["enero", "febrero", "marzo", "abril", "mayo", "junio", "julio", "agosto", "septiembre",
|
|
"setiembre", "octubre", "noviembre", "diciembre"],
|
|
["hoy", "mañana", "ayer", "anoche"],
|
|
],
|
|
noComma: ["y", "o", "pero", "que", "a", "de", "en", "con", "para", "por", "el", "la", "los", "las",
|
|
"un", "una", "mi", "tu", "su", "porque", "si"]
|
|
)
|
|
|
|
private static let german = Rules(
|
|
fillers: [],
|
|
markers: [["nein"], ["nein", "warte"], ["ich", "meine"], ["besser", "gesagt"], ["sorry"],
|
|
["entschuldigung"]],
|
|
plainMarkers: [["besser", "gesagt"]],
|
|
kinds: [
|
|
["zwei", "drei", "vier", "fünf", "sechs", "sieben", "acht", "neun", "zehn", "elf", "zwölf"],
|
|
["montag", "dienstag", "mittwoch", "donnerstag", "freitag", "samstag", "sonntag"],
|
|
["januar", "jänner", "februar", "märz", "april", "mai", "juni", "juli", "august", "september",
|
|
"oktober", "november", "dezember"],
|
|
["heute", "morgen", "gestern", "übermorgen"],
|
|
],
|
|
noComma: ["und", "oder", "aber", "dass", "der", "die", "das", "ein", "eine", "mit", "zu", "in",
|
|
"an", "auf", "für", "von", "ich", "du", "er", "sie", "wir"]
|
|
)
|
|
|
|
private static let norwegian = Rules(
|
|
fillers: [],
|
|
markers: [["nei"], ["nei", "vent"], ["jeg", "mener"], ["rettere", "sagt"], ["unnskyld"], ["sorry"]],
|
|
plainMarkers: [["rettere", "sagt"]],
|
|
kinds: [
|
|
["to", "tre", "fire", "fem", "seks", "sju", "syv", "åtte", "ni", "ti", "elleve", "tolv"],
|
|
["mandag", "tirsdag", "onsdag", "torsdag", "fredag", "lørdag", "søndag"],
|
|
["januar", "februar", "mars", "april", "mai", "juni", "juli", "august", "september",
|
|
"oktober", "november", "desember"],
|
|
],
|
|
noComma: ["og", "eller", "men", "at", "jeg", "du", "han", "hun", "vi", "en", "et", "den", "det",
|
|
"på", "i", "til", "med", "for", "som"]
|
|
)
|
|
}
|
|
|
|
private extension Array {
|
|
subscript(safe index: Int) -> Element? {
|
|
indices.contains(index) ? self[index] : nil
|
|
}
|
|
}
|