Skip to content

Commit 563ff5e

Browse files
fixing more failing tests in __tests__/spelling.js
1 parent 7cdf8c3 commit 563ff5e

2 files changed

Lines changed: 5 additions & 2 deletions

File tree

__tests__/spelling-ignore.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,9 @@
239239
- substring
240240
- initialisms
241241
- sublist
242+
- tokenize
243+
- tokenized
244+
- subsequence
242245

243246
# Parts of Unicode
244247
- 000A

pages/glossary/label-in-name-algorithm.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@ Sub-algorithm to tokenize a string:
1616

1717
- Convert the string to lower case.
1818
- For each character that either a) represents non-text content, or b) isn't a letter or a digit: replace that character with a space character.
19-
- For a) Judgement of "non-text" probably can't be fully automated. eg. "X" for "close" probably can be, but presumably there are more cases than this.
20-
- For b) Use the unicode classes Letter, Mark, and "Number, Decimal Digit [Nd]". (This will exclude hyphens, punctuation, emoji, and more.)
19+
- For a) Judgment of "non-text" probably can't be fully automated. eg. "X" for "close" probably can be, but presumably there are more cases than this.
20+
- For b) Use the Unicode classes Letter, Mark, and "Number, Decimal Digit [Nd]". (This will exclude hyphens, punctuation, emoji, and more.)
2121
- Remove all characters that are within parentheses (AKA round brackets).
2222
- Ignore square brackets and braces.
2323
- Split the string into a list of strings, using a whitespace regular expression as the separator.

0 commit comments

Comments
 (0)