Go to file
Lukhnos Liu d064f420e4 Use a parseless phrase db to speed up LM loading
We take advantage of the fact that no one is able to modify the phrase
databases shipped with the binary (guranteed by macOS's integrity check
for notarized apps), and we can simply pre-sort the phrases in the
database files.

With this change, we can speed up McBopomofo's language model loading
during the app initialization by about 500-800x on a 2018 Intel MacBook
Pro. The LM loading used to take 300-400 ms, but now it's done within a
sub-millisecond range (0.5-0.6 ms). Microbenchmarking shows that
ParselessLM is about 16000x faster than FastLM. We amortize the latency
during the query time, and even by deferring the parsing, ParselessLM is
only ~1.5x slower than FastLM, and both LM classes serve queries unedr 6
microseconds (that's 0.006 ms), which means the tradeoff only
contributes to neglible overall latency.

This PR requires some small changes to the phrase db cooking scripts.
Python 3 is now used and the (value, reading, score) tuples are
rearranged to (reading, value, score) and sorted by reading ("key"). A
header is added to the phrase databases to call out the fact that these
are pre-sorted.

clang-format is used to apply WebKit C++ style to the new code. This
also applies to KeyValueBlobReader that was added recently.

Microbenchmark result below:

```
---------------------------------------------------------------------
Benchmark                           Time             CPU   Iterations
---------------------------------------------------------------------
BM_ParselessLMOpenClose         17710 ns        17199 ns        33422
BM_FastLMOpenClose          376520248 ns    367526500 ns            2
BM_ParselessLMFindUnigrams       5967 ns         5899 ns       113729
BM_FastLMFindUnigrams            2268 ns         2265 ns       307038
```
2022-01-15 16:15:02 -08:00
.github/workflows Updates CI settings. 2022-01-02 22:13:14 +08:00
McBopomofo.xcodeproj Use a parseless phrase db to speed up LM loading 2022-01-15 16:15:02 -08:00
Packages Brings back VXHanConvert. 2022-01-14 22:15:17 +08:00
Source Use a parseless phrase db to speed up LM loading 2022-01-15 16:15:02 -08:00
.gitignore Fixes duplicated code and typos. 2022-01-11 13:46:29 +08:00
.travis.yml speed up travis-ci build 2016-06-10 12:54:02 +08:00
LICENSE.txt Add LICENSE.txt 2021-11-24 10:04:29 -08:00
README.markdown Also updates the README file. 2022-01-14 02:34:23 +08:00

README.markdown

OpenVanilla McBopomofo 小麥注音輸入法

系統需求

小麥注音輸入法可以在 macOS 10.10 以上版本運作。如果您要自行編譯小麥注音輸入法,或參與開發,您需要:

  • macOS 10.15 Catalina 以上版本
  • Xcode 12.0 以上版本

由於小麥注音輸入法開始使用 SPMSwift Package Manager管理相依套件因此必須使用 Xcode 12 開發。

開發流程

用 Xcode 開啟 McBopomofo.xcodeproj,選 "McBopomofo Installer" targetbuild 完之後直接執行該安裝程式,就可以安裝小麥注音。

第一次安裝完,日後程式碼或詞庫有任何修改,只要重複上述流程,再次安裝小麥注音即可。

要注意的是 macOS 可能會限制同一次 login session 能 kill 同一個輸入法 process 的次數(安裝程式透過 kill input method process 來讓新版的輸入法生效)。如果安裝若干次後,發現程式修改的結果並沒有出現,或甚至輸入法已無法再選用,只要登出目前帳號再重新登入即可。

軟體授權

本專案採用 MIT License 釋出,使用者可自由使用、散播本軟體,惟散播時必須完整保留版權聲明及軟體授權(詳全文)。