#text #unicode #grapheme #word #boundary

no-std unicode-segmentation

This crate provides Grapheme Cluster and Word boundaries according to Unicode Standard Annex #29 rules

11 releases (6 stable)

1.2.1 Apr 29, 2018
1.2.0 May 16, 2017
1.1.0 Feb 6, 2017
1.0.1 Dec 27, 2016
0.1.1 Jul 9, 2015

#11 in Text processing

Download history 11495/week @ 2018-09-19 14095/week @ 2018-09-26 14856/week @ 2018-10-03 14533/week @ 2018-10-10 16019/week @ 2018-10-17 14768/week @ 2018-10-24 16513/week @ 2018-10-31 20091/week @ 2018-11-07 20281/week @ 2018-11-14 21152/week @ 2018-11-21 21339/week @ 2018-11-28 23395/week @ 2018-12-05 22654/week @ 2018-12-12

45,832 downloads per month
Used in 582 crates (63 directly)

MIT/Apache

389KB
5K SLoC

Iterators which split strings on Grapheme Cluster or Word boundaries, according to the Unicode Standard Annex #29 rules.

Build Status

Documentation

extern crate unicode_segmentation;

use unicode_segmentation::UnicodeSegmentation;

fn main() {
    let s = "a̐éö̲\r\n";
    let g = UnicodeSegmentation::graphemes(s, true).collect::<Vec<&str>>();
    let b: &[_] = &["", "", "ö̲", "\r\n"];
    assert_eq!(g, b);

    let s = "The quick (\"brown\") fox can't jump 32.3 feet, right?";
    let w = s.unicode_words().collect::<Vec<&str>>();
    let b: &[_] = &["The", "quick", "brown", "fox", "can't", "jump", "32.3", "feet", "right"];
    assert_eq!(w, b);

    let s = "The quick (\"brown\")  fox";
    let w = s.split_word_bounds().collect::<Vec<&str>>();
    let b: &[_] = &["The", " ", "quick", " ", "(", "\"", "brown", "\"", ")", " ", " ", "fox"];
    assert_eq!(w, b);
}

no_std

unicode-segmentation does not depend on libstd, so it can be used in crates with the #![no_std] attribute.

crates.io

You can use this package in your project by adding the following to your Cargo.toml:

[dependencies]
unicode-segmentation = "1.1.0"

Change Log

1.2.0

  • New GraphemeCursor API allows random access and bidirectional iteration.
  • Fixed incorrect splitting of certain emoji modifier sequences.

1.1.0

  • Add as_str methods to the iterator types.

1.0.3

  • Code cleanup and additional tests.

1.0.1

  • Fix a bug affecting some grapheme clusters containing Prepend characters.

1.0.0

  • Upgrade to Unicode 9.0.0.

No runtime deps