From fa9c0f5373894823ab87a31ddb70aa61cf16c031 Mon Sep 17 00:00:00 2001 From: David Oti Date: Wed, 19 Aug 2026 10:10:52 +0100 Subject: [PATCH 1/7] Modernize codebase for PHP 8.1+, update dependencies, and improve error handling; add migration guide and static analysis support. --- .gitattributes | 1 + .github/workflows/ci.yml | 3 + CHANGELOG.md | 36 +++++++ KNOWN-DIVERGENCES.md | 26 +++-- MIGRATION.md | 106 +++++++++++++++++++++ README.md | 7 +- composer.json | 6 +- phpstan.neon | 11 +++ phpunit.xml | 9 +- src/Analyzer.php | 103 ++++++++------------ src/Config/Config.php | 2 +- src/Exceptions/InvalidLexiconException.php | 19 ++++ src/Procedures/SentiText.php | 35 +++++-- tests/ApiContractTest.php | 63 ++++++------ tools/test-matrix.sh | 4 +- 15 files changed, 298 insertions(+), 133 deletions(-) create mode 100644 MIGRATION.md create mode 100644 phpstan.neon create mode 100644 src/Exceptions/InvalidLexiconException.php diff --git a/.gitattributes b/.gitattributes index dedba7b..e77395c 100644 --- a/.gitattributes +++ b/.gitattributes @@ -4,6 +4,7 @@ /tests/ export-ignore /tools/ export-ignore /phpunit.xml export-ignore +/phpstan.neon export-ignore /.gitattributes export-ignore /.gitignore export-ignore /KNOWN-DIVERGENCES.md export-ignore diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5ce6b70..4cccab6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -38,6 +38,9 @@ jobs: - name: Run characterization suite run: composer test + - name: Static analysis + run: composer stan + # The baseline must be reproducible: regenerating it on a clean checkout # must produce no diff. A diff here means scoring is not deterministic # across PHP versions, which would invalidate the v2.0 parity guarantee. diff --git a/CHANGELOG.md b/CHANGELOG.md index 3dbe471..9ba2f21 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,42 @@ All notable changes to this project are documented here. This project follows [Semantic Versioning](https://semver.org/). +## [2.0.0] - unreleased + +### Scores are unchanged + +**2.0.0 produces byte-identical scores to 1.3.0** across all 355 pinned cases, +verified on PHP 8.1–8.4 in CI. This release modernizes the codebase; it does not +touch the scoring path. See `MIGRATION.md`. + +### Changed — BREAKING + +- **PHP 8.1+ is now required** (`^8.1`). Users on older runtimes stay on `1.x`, + which remains supported. +- Twelve internal methods are now `private`: `IsNegated`, `make_lex_dict`, + `make_emoji_dict`, `score_valence`, `_least_check`, `_but_check`, + `_idioms_check`, `_never_check`, `_punctuation_emphasis`, `_amplify_ep`, + `_amplify_qm`, `_sift_sentiment_scores`. +- `SentiText` is `@internal`; its public properties are now private with + `getWordsAndEmoticons()` / `isCapDifferential()` accessors. +- A missing lexicon file throws `Sentiment\Exceptions\InvalidLexiconException` + instead of calling `die()`. +- Removed `_sentiment_laden_idioms_check()`, which was public and never called. + No behavioural change — it is why `SENTIMENT_LADEN_IDIOMS` never fired. + +### Fixed + +- Dynamic property creation (`$emoji_lexicon`, `$emojis`), deprecated since PHP + 8.2, now declared. The package emits no deprecation notices, enforced by + `failOnDeprecation="true"`. + +### Added + +- Full parameter, return and property types across `Analyzer`, `SentiText` and + `Config`. +- PHPStan at level 5, wired into CI. +- `MIGRATION.md`. + ## [1.3.0] - 2026-08-19 ### Fixed diff --git a/KNOWN-DIVERGENCES.md b/KNOWN-DIVERGENCES.md index 6020565..20bdc57 100644 --- a/KNOWN-DIVERGENCES.md +++ b/KNOWN-DIVERGENCES.md @@ -85,24 +85,20 @@ they cannot work until the idiom matcher does. --- -## 3. Dynamic property deprecations (PHP 8.2+) +## 3. Dynamic property deprecations (PHP 8.2+) — FIXED in 2.0.0 -`Analyzer::__construct()` assigns `$this->emoji_lexicon` and `$this->emojis` -without declaring them (`src/Analyzer.php:25` and `:27`). Deprecated since PHP -8.2; two notices fire on every instantiation. +`Analyzer::__construct()` assigned `$this->emoji_lexicon` and `$this->emojis` +without declaring them (`src/Analyzer.php`). Deprecated since PHP 8.2; two +notices fired on every instantiation. PHP 8.1 emitted nothing, as dynamic +properties were not deprecated there. -**PHP 8.1 emits nothing** — dynamic properties are not deprecated there. The -suite therefore reports 368 passing on 8.2–8.4 and 367 passing plus 1 skipped on -8.1, which is expected, not a gap. +Both are now declared and typed. `phpunit.xml` sets `failOnDeprecation="true"`, +so any new deprecation fails the build, and the +`testKnownDynamicPropertyDeprecationsStillPresent()` tripwire has been removed — +it had done its job. -`phpunit.xml` therefore sets `failOnDeprecation="false"`. - -**Scheduled for v2.0**, where declaring the properties is part of the -modernization. `ApiContractTest::testKnownDynamicPropertyDeprecationsStillPresent()` -fails once they are declared — that failure is the signal to flip -`failOnDeprecation` to `true` and delete the test. - ---- +Fixed on the `2.x` line only. The `1.x` maintenance line still emits these +notices on PHP 8.2+, which is expected: it exists to support PHP < 8.1. ## 4. VADER lexicon file carries a UTF-8 BOM diff --git a/MIGRATION.md b/MIGRATION.md new file mode 100644 index 0000000..80e7a99 --- /dev/null +++ b/MIGRATION.md @@ -0,0 +1,106 @@ +# Migrating from 1.x to 2.0 + +**Your sentiment scores do not change.** 2.0 produces byte-identical `neg`, +`neu`, `pos` and `compound` values to `1.3.0` across the full 355-case +characterization suite, verified on PHP 8.1, 8.2, 8.3 and 8.4. This is enforced +in CI, not asserted by hand. + +If you are upgrading from `1.2.2` or earlier, scores **do** change — but that +belongs to `1.3.0`, not to 2.0. See the "Coming from 1.2.2 or earlier" section. + +--- + +## 1. PHP 8.1 is required + +```json +"require": { "php": "^8.1" } +``` + +This is the breaking change that matters. Composer will not resolve 2.0 on an +older runtime, so no amount of API compatibility helps there. + +**If you cannot upgrade PHP:** stay on `1.x`. It remains installable, receives +security fixes, and carries the same test suite. `composer require +davmixcool/php-sentiment-analyzer:^1.3` pins you to it. + +## 2. What has NOT changed + +These are frozen and enforced by `tests/ApiContractTest.php`: + +```php +$analyzer = new Analyzer(); // same two optional path arguments +$scores = $analyzer->getSentiment($text); // same ['neg','neu','pos','compound'] +$analyzer->updateLexicon(['rubbish' => -1.5]); // same lowercasing, same coercion +``` + +- `getSentiment()` returns the **same plain array**, same keys, same order, same + rounding. It does not return an object. +- `updateLexicon()` keeps lowercasing keys, keeps coercing non-numeric values to + `0`, and keeps ignoring non-array input. +- The constructor keeps resolving lexicon paths relative to the package's `src/` + directory. + +## 3. Accepted breaks + +### 3.1 Internal methods are now private + +Twelve methods that were `public` but plainly internal are now `private`: + +`IsNegated`, `make_lex_dict`, `make_emoji_dict`, `score_valence`, +`_least_check`, `_but_check`, `_idioms_check`, `_never_check`, +`_punctuation_emphasis`, `_amplify_ep`, `_amplify_qm`, `_sift_sentiment_scores` + +They were never part of the intended API — they were internals of the VADER port +that happened to be reachable. If you called one directly, open an issue +describing what for; that is a real use case worth designing an API around. + +### 3.2 `_sentiment_laden_idioms_check()` has been removed + +It was `public`, and it was **never called** — zero call sites in 1.x. Its +absence is exactly why the 12 `SENTIMENT_LADEN_IDIOMS` entries never affected +any score. Removing it changes no behaviour; it only stops the code implying a +feature that does not exist. The underlying divergence remains and is documented +in `KNOWN-DIVERGENCES.md` §2. + +### 3.3 `SentiText` is encapsulated + +`SentiText::$words_and_emoticons` and `$is_cap_diff` were public properties and +are now private, with `getWordsAndEmoticons()` and `isCapDifferential()` +accessors. The class is marked `@internal`. + +### 3.4 A missing lexicon file now throws instead of calling `die()` + +```php +use Sentiment\Exceptions\InvalidLexiconException; + +try { + $analyzer = new Analyzer('Lexicons/custom.txt'); +} catch (InvalidLexiconException $e) { + // handle it +} +``` + +v1 called `die()`, terminating the host process — behaviour a library should +never impose on the application embedding it. If you passed a custom lexicon +path and relied on the process dying, you now need a `catch`. + +## 4. Coming from 1.2.2 or earlier + +`1.3.0` fixed a defect in `_never_check()` that zeroed the sentiment of any word +within two tokens of "so" or "this": + +| Input | 1.2.2 | 1.3.0 and 2.0 | +|---|---|---| +| `this is good` | 0.0000 | +0.4404 | +| `this is bad` | 0.0000 | -0.5423 | +| `so good` | 0.0000 | +0.4877 | + +**Re-score any stored text** containing those words near sentiment terms. This +change is attributable to `1.3.0`; 2.0 inherits it unchanged. + +## 5. What has not been fixed + +2.0 is a modernization, not a scoring release. Known divergences from reference +VADER — most notably **15 of 21 idioms that never fire** — are reproduced +exactly and remain documented in `KNOWN-DIVERGENCES.md`. Fixing them will be its +own release with its own changelog entry, because it changes output. diff --git a/README.md b/README.md index a3a8104..26860ba 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,11 @@ PHP Sentiment Analyzer is a lexicon and rule-based sentiment analysis tool that ## Requirements -* PHP 5.5 and above +* PHP 8.1 and above + +> Using PHP below 8.1? Install the `1.x` line instead — it is maintained and +> produces the same scores: +> `composer require davmixcool/php-sentiment-analyzer:^1.3` ## Contents @@ -28,6 +32,7 @@ PHP Sentiment Analyzer is a lexicon and rule-based sentiment analysis tool that ## Documentation * [Changelog](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/CHANGELOG.md) — release history, including scoring changes +* [Migrating from 1.x to 2.0](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/MIGRATION.md) — breaking changes, and why your scores do not move * [Known divergences](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/KNOWN-DIVERGENCES.md) — behaviour that differs from reference VADER, documented and pinned by the test suite ### Install diff --git a/composer.json b/composer.json index fbcd18a..cf6ae35 100644 --- a/composer.json +++ b/composer.json @@ -21,9 +21,10 @@ } ], "require": { - "php": ">=5.5.9" + "php": "^8.1" }, "require-dev": { + "phpstan/phpstan": "^2.2", "phpunit/phpunit": "^10.5" }, "autoload": { @@ -40,7 +41,8 @@ "test": "phpunit", "baseline": "php tools/generate-baseline.php", "matrix": "./tools/test-matrix.sh", - "matrix:fresh": "./tools/test-matrix.sh --fresh" + "matrix:fresh": "./tools/test-matrix.sh --fresh", + "stan": "phpstan analyse --no-progress" }, "config": { "sort-packages": true diff --git a/phpstan.neon b/phpstan.neon new file mode 100644 index 0000000..34d6500 --- /dev/null +++ b/phpstan.neon @@ -0,0 +1,11 @@ +parameters: + level: 5 + paths: + - src + - tests + - tools + + # Level 6 currently reports ~48 "no value type specified in iterable type" + # findings. Those need generic array annotations throughout the scoring + # path, which is worth doing — but as its own change, not folded into the + # modernization, where it would obscure the diff that must prove parity. diff --git a/phpunit.xml b/phpunit.xml index a6d93b0..08fd42a 100644 --- a/phpunit.xml +++ b/phpunit.xml @@ -9,12 +9,11 @@ displayDetailsOnTestsThatTriggerDeprecations="true" failOnWarning="true" failOnNotice="true" - failOnDeprecation="false"> + failOnDeprecation="true"> diff --git a/src/Analyzer.php b/src/Analyzer.php index 4a1a103..f53f182 100644 --- a/src/Analyzer.php +++ b/src/Analyzer.php @@ -3,6 +3,7 @@ namespace Sentiment; use Sentiment\Config\Config; +use Sentiment\Exceptions\InvalidLexiconException; use Sentiment\Procedures\SentiText; /* @@ -11,12 +12,19 @@ class Analyzer { - private $lexicon_file = ""; - private $lexicon = ""; + private string $lexicon_file = ""; - private $current_sentitext = null; + private string $emoji_lexicon = ""; - public function __construct($lexicon_file = "Lexicons/vader_sentiment_lexicon.txt",$emoji_lexicon='Lexicons/emoji_utf8_lexicon.txt') + /** @var array Term => valence, as read from the lexicon file. */ + private array $lexicon = []; + + /** @var array Emoji character => description. */ + private array $emojis = []; + + private ?SentiText $current_sentitext = null; + + public function __construct(string $lexicon_file = "Lexicons/vader_sentiment_lexicon.txt", string $emoji_lexicon = 'Lexicons/emoji_utf8_lexicon.txt') { //Not sure about this as it forces lexicon file to be in the same directory as executing script $this->lexicon_file = __DIR__ . DIRECTORY_SEPARATOR . $lexicon_file; @@ -30,7 +38,7 @@ public function __construct($lexicon_file = "Lexicons/vader_sentiment_lexicon.tx /* Determine if input contains negation words */ - public function IsNegated($wordToTest, $include_nt = true) + private function IsNegated(string $wordToTest, bool $include_nt = true): bool { $wordToTest = strtolower($wordToTest); if (in_array($wordToTest, Config::NEGATE)) { @@ -49,30 +57,28 @@ public function IsNegated($wordToTest, $include_nt = true) /* Convert lexicon file to a dictionary */ - public function make_lex_dict() + private function make_lex_dict(): array { $lex_dict = []; - $fp = fopen($this->lexicon_file, "r"); + $fp = @fopen($this->lexicon_file, "r"); if (!$fp) { - die("Cannot load lexicon file"); + throw InvalidLexiconException::unreadable($this->lexicon_file); } while (($line = fgets($fp, 4096)) !== false) { list($word, $measure) = explode("\t", trim($line)); - //.strip().split('\t')[0:2] - $lex_dict[$word] = $measure; - //lex_dict[word] = float(measure) + $lex_dict[$word] = (float) $measure; } return $lex_dict; } - public function make_emoji_dict() { + private function make_emoji_dict(): array { $emoji_dict = []; - $fp = fopen($this->emoji_lexicon, "r"); + $fp = @fopen($this->emoji_lexicon, "r"); if (!$fp) { - die("Cannot load emoji lexicon file"); + throw InvalidLexiconException::unreadable($this->emoji_lexicon); } while (($line = fgets($fp, 4096)) !== false) { @@ -87,18 +93,12 @@ public function make_emoji_dict() { public function updateLexicon($arr) { if(!is_array($arr)) return []; - $lexicon = []; foreach ($arr as $word => $valence) { $this->lexicon[strtolower($word)] = is_numeric($valence)? $valence : 0; } } - private function IsKindOf($firstWord, $secondWord) - { - return "kind" === strtolower($firstWord) && "of" === strtolower($secondWord); - } - - private function IsBoosterWord($word) + private function IsBoosterWord(string $word): bool { return array_key_exists(strtolower($word), Config::BOOSTER_DICT); } @@ -108,14 +108,14 @@ private function getBoosterScaler($word) return Config::BOOSTER_DICT[strtolower($word)]; } - private function IsInLexicon($word) + private function IsInLexicon(string $word): bool { $lowercase = strtolower($word); return array_key_exists($lowercase, $this->lexicon); } - private function IsUpperCaseWord($word) + private function IsUpperCaseWord(string $word): bool { return ctype_upper($word); } @@ -125,7 +125,7 @@ private function getValenceFromLexicon($word) return $this->lexicon[strtolower($word)]; } - private function getTargetWordFromContext($wordInContext) + private function getTargetWordFromContext(array $wordInContext): string { return $wordInContext[count($wordInContext)-1]; } @@ -133,7 +133,7 @@ private function getTargetWordFromContext($wordInContext) /* Gets the precedding two words to check for emphasis */ - private function getWordInContext($wordList, $currentWordPosition) + private function getWordInContext(array $wordList, int $currentWordPosition): array { $precedingWordList =[]; @@ -166,7 +166,7 @@ private function getWordInContext($wordList, $currentWordPosition) Positive values are positive valence, negative value are negative valence. */ - public function getSentiment($text) + public function getSentiment(string $text): array { $text_no_emoji = ''; @@ -191,7 +191,7 @@ public function getSentiment($text) $this->current_sentitext = new SentiText($text); $sentiments = []; - $words_and_emoticons = $this->current_sentitext->words_and_emoticons; + $words_and_emoticons = $this->current_sentitext->getWordsAndEmoticons(); for ($i=0; $i<=count($words_and_emoticons)-1; $i++) { $valence = 0.0; @@ -223,22 +223,16 @@ public function getSentiment($text) } - private function str_split_unicode($str, $l = 0) { - if ($l > 0) { - $ret = array(); - $len = mb_strlen($str, "UTF-8"); - for ($i = 0; $i < $len; $i += $l) { - $ret[] = mb_substr($str, $i, $l, "UTF-8"); - } - return $ret; - } + /** @return array */ + private function str_split_unicode(string $str): array + { return preg_split("//u", $str, -1, PREG_SPLIT_NO_EMPTY); } private function applyValenceCapsBoost($targetWord, $valence) { - if ($this->IsUpperCaseWord($targetWord) && $this->current_sentitext->is_cap_diff) { + if ($this->IsUpperCaseWord($targetWord) && $this->current_sentitext->isCapDifferential()) { if ($valence > 0) { $valence += Config::C_INCR; } else { @@ -332,7 +326,7 @@ private function modifyValenceBasedOnContext($wordInContext, $valence) return $valence; } - public function _least_check($wordInContext, $valence) + private function _least_check($wordInContext, $valence) { // check for negation case using "least" //if the previous word is least" @@ -346,7 +340,7 @@ public function _least_check($wordInContext, $valence) return $valence; } - public function _but_check($words_and_emoticons, $sentiments) + private function _but_check(array $words_and_emoticons, array $sentiments): array { // check for modification in sentiment due to contrastive conjunction 'but' $bi = array_search("but", $words_and_emoticons); @@ -366,7 +360,7 @@ public function _but_check($words_and_emoticons, $sentiments) return $sentiments; } - public function _idioms_check($wordInContext, $valence) + private function _idioms_check($wordInContext, $valence) { $onezero = sprintf("%s %s", $wordInContext[2], $wordInContext[3]); @@ -416,7 +410,7 @@ public function _idioms_check($wordInContext, $valence) return $valence; } - public function _never_check($wordInContext, $valance) + private function _never_check($wordInContext, $valance) { //If the sentiment word is preceded by never so/this we apply a modifier $neverModifier = 0; @@ -444,25 +438,8 @@ public function _never_check($wordInContext, $valance) return $valance; } - public function _sentiment_laden_idioms_check($valence, $senti_text_lower){ - # Future Work - # check for sentiment laden idioms that don't contain a lexicon word - $idioms_valences = []; - foreach (Config::SENTIMENT_LADEN_IDIOMS as $idiom) { - if(in_array($idiom, $senti_text_lower)){ - //print($idiom, $senti_text_lower) - $valence = Config::SENTIMENT_LADEN_IDIOMS[$idiom]; - $idioms_valences[] = $valence; - } - } - - if ((strlen($idioms_valences) > 0)) { - $valence = ( array_sum( explode( ',', $idioms_valences ) ) / floatval(strlen($idioms_valences))); - } - return $valence; - } - public function _punctuation_emphasis($sum_s, $text) + private function _punctuation_emphasis($sum_s, string $text): float { // add emphasis from exclamation points and question marks $ep_amplifier = $this->_amplify_ep($text); @@ -472,7 +449,7 @@ public function _punctuation_emphasis($sum_s, $text) return $punct_emph_amplifier; } - public function _amplify_ep($text) + private function _amplify_ep(string $text): float { // check for added emphasis resulting from exclamation points (up to 4 of them) $ep_count = substr_count($text, "!"); @@ -486,7 +463,7 @@ public function _amplify_ep($text) return $ep_amplifier; } - public function _amplify_qm($text) + private function _amplify_qm(string $text): float { # check for added emphasis resulting from question marks (2 or 3+) $qm_count = substr_count($text, "?"); @@ -504,7 +481,7 @@ public function _amplify_qm($text) return $qm_amplifier; } - public function _sift_sentiment_scores($sentiments) + private function _sift_sentiment_scores(array $sentiments): array { # want separate positive versus negative sentiment scores $pos_sum = 0.0; @@ -525,7 +502,7 @@ public function _sift_sentiment_scores($sentiments) return [$pos_sum, $neg_sum, $neu_count]; } - public function score_valence($sentiments, $text) + private function score_valence(array $sentiments, string $text): array { if ($sentiments) { $sum_s = array_sum($sentiments); diff --git a/src/Config/Config.php b/src/Config/Config.php index 05f0304..6a7f24e 100644 --- a/src/Config/Config.php +++ b/src/Config/Config.php @@ -67,7 +67,7 @@ class Config Normalize the score to be between -1 and 1 using an alpha that approximates the max expected value */ - public static function normalize($score, $alpha = 15) + public static function normalize(float $score, int $alpha = 15): float { $norm_score = $score/sqrt(($score*$score) + $alpha); return $norm_score; diff --git a/src/Exceptions/InvalidLexiconException.php b/src/Exceptions/InvalidLexiconException.php new file mode 100644 index 0000000..bb61d27 --- /dev/null +++ b/src/Exceptions/InvalidLexiconException.php @@ -0,0 +1,19 @@ + */ + private array $words_and_emoticons = []; + + private bool $is_cap_diff = false; const PUNC_LIST = [".", "!", "?", ",", ";", ":", "-", "'", "\"", "!!", "!!!", "??", "???", "?!?", "!?!", "?!?!", "!?!?"]; - function __construct($text) + public function __construct(string $text) { //checking that is string //if (!isinstance(text, str)){ @@ -30,16 +36,27 @@ function __construct($text) $this->is_cap_diff = $this->allcap_differential($this->words_and_emoticons); } + /** @return array */ + public function getWordsAndEmoticons(): array + { + return $this->words_and_emoticons; + } + + public function isCapDifferential(): bool + { + return $this->is_cap_diff; + } + /* Remove all punctation from a string */ - function strip_punctuation($string) + public function strip_punctuation(string $string): string { //$string = strtolower($string); return preg_replace("/[[:punct:]]+/", "", $string); } - function array_count_values_of($haystack, $needle) + public function array_count_values_of(array $haystack, string $needle): int { if (!in_array($needle, $haystack, true)) { return 0; @@ -54,7 +71,7 @@ function array_count_values_of($haystack, $needle) :param list words: The words to inspect :returns: `True` if some but not all items in `words` are ALL CAPS */ - private function allcap_differential($words) + private function allcap_differential(array $words): bool { $is_different = false; @@ -72,7 +89,7 @@ private function allcap_differential($words) return $is_different; } - function _words_only() + public function _words_only(): array { $text_mod = $this->strip_punctuation($this->text); // removes punctuation (but loses emoticons & contractions) @@ -84,7 +101,7 @@ function _words_only() return $words_only; } - function _words_and_emoticons() + public function _words_and_emoticons(): array { $wes = preg_split('/\s+/', $this->text); diff --git a/tests/ApiContractTest.php b/tests/ApiContractTest.php index d5d972c..cd7f2a9 100644 --- a/tests/ApiContractTest.php +++ b/tests/ApiContractTest.php @@ -5,6 +5,7 @@ use PHPUnit\Framework\TestCase; use ReflectionClass; use Sentiment\Analyzer; +use Sentiment\Exceptions\InvalidLexiconException; /** * Pins the frozen backward-compatibility surface from PRD §3. @@ -26,10 +27,17 @@ public function testGetSentimentReturnsFrozenKeysInOrder(): void public function testGetSentimentReturnsPlainArrayNotAnObject(): void { - // PRD §3: getSentiment() must NOT return SentimentResult in v2, and - // SentimentResult must NOT implement ArrayAccess. This test is the - // tripwire for that decision. - $this->assertIsArray((new Analyzer())->getSentiment('This is good.')); + // PRD §3: getSentiment() must NOT return SentimentResult, and + // SentimentResult must NOT implement ArrayAccess. Assert the DECLARED + // return type rather than the runtime value — that is what callers + // and static analysis actually depend on, and a change to an object + // type would be caught here even if it were array-like at runtime. + $returnType = (new ReflectionClass(Analyzer::class)) + ->getMethod('getSentiment') + ->getReturnType(); + + $this->assertNotNull($returnType, 'getSentiment() must declare a return type'); + $this->assertSame('array', (string) $returnType); } public function testGetSentimentValuesAreNumeric(): void @@ -124,6 +132,22 @@ public function testConstructorResolvesPathsRelativeToSrcDirectory(): void ); } + public function testMissingLexiconThrowsInsteadOfKillingTheProcess(): void + { + // v1 called die() here, terminating the host application. v2 throws. + // Accepted break, documented in MIGRATION.md. + $this->expectException(InvalidLexiconException::class); + + new Analyzer('Lexicons/does-not-exist.txt'); + } + + public function testMissingEmojiLexiconThrows(): void + { + $this->expectException(InvalidLexiconException::class); + + new Analyzer('Lexicons/vader_sentiment_lexicon.txt', 'Lexicons/nope.txt'); + } + public function testNoRuntimeCodePathPerformsNetworkIo(): void { // PRD: "Runtime inference must never require a network connection." @@ -153,35 +177,4 @@ public function testNoRuntimeCodePathPerformsNetworkIo(): void } } - public function testKnownDynamicPropertyDeprecationsStillPresent(): void - { - // Documents the two deprecations catalogued in KNOWN-DIVERGENCES.md. - // When v2.0 declares these properties this test FAILS — that is the - // signal to flip failOnDeprecation to "true" in phpunit.xml and delete - // this test. It exists so the cleanup cannot be forgotten. - if (PHP_VERSION_ID < 80200) { - $this->markTestSkipped('Dynamic properties are only deprecated from PHP 8.2.'); - } - - $deprecations = []; - - set_error_handler( - static function (int $severity, string $message) use (&$deprecations): bool { - $deprecations[] = $message; - return true; - }, - E_DEPRECATED - ); - - try { - new Analyzer(); - } finally { - restore_error_handler(); - } - - $joined = implode("\n", $deprecations); - - $this->assertStringContainsString('emoji_lexicon', $joined); - $this->assertStringContainsString('emojis', $joined); - } } diff --git a/tools/test-matrix.sh b/tools/test-matrix.sh index 5546e7a..5c7e829 100755 --- a/tools/test-matrix.sh +++ b/tools/test-matrix.sh @@ -202,7 +202,7 @@ for version in "${VERSIONS[@]}"; do # Install + test + generate in one container (see docker_fresh). if out="$(docker_fresh "$image" "$stage" "$version" 2>&1)"; then printf 'install: ok ' - summary="$(grep -E '^Tests:' <<<"$out" | tail -1 || true)" + summary="$(grep -E '^(OK|Tests:)' <<<"$out" | tail -1 || true)" printf 'tests: %-46s ' "${summary:-OK}" else if grep -q '###PHPUNIT###' <<<"$out"; then @@ -219,7 +219,7 @@ for version in "${VERSIONS[@]}"; do else # 1. Test suite if test_output="$(docker_run "$image" "$stage" php vendor/bin/phpunit 2>&1)"; then - summary="$(grep -E '^Tests:' <<<"$test_output" | tail -1 || true)" + summary="$(grep -E '^(OK|Tests:)' <<<"$test_output" | tail -1 || true)" printf 'tests: %-46s ' "${summary:-OK}" else printf 'tests: FAILED\n' From 14aca91bba3d90b98b2ab3f49da634e24de5f1f2 Mon Sep 17 00:00:00 2001 From: David Oti Date: Wed, 19 Aug 2026 10:20:40 +0100 Subject: [PATCH 2/7] Update actions/checkout to v7 in CI workflow --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4cccab6..03290e9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -20,7 +20,7 @@ jobs: php: ['8.1', '8.2', '8.3', '8.4'] steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7 - name: Setup PHP uses: shivammathur/setup-php@v2 From c9a3e90275bd9bbfa9221113d7b8552092daf667 Mon Sep 17 00:00:00 2001 From: David Oti Date: Wed, 19 Aug 2026 10:45:09 +0100 Subject: [PATCH 3/7] Implement new API with immutable SentimentResult and analyze methods; update documentation and migration guide --- CHANGELOG.md | 18 +++ MIGRATION.md | 44 +++++ README.md | 52 ++++++ src/Analyzer.php | 77 +++++++++ .../InvalidLexiconTermException.php | 36 +++++ src/SentimentResult.php | 115 +++++++++++++ tests/CharacterizationTest.php | 36 +++++ tests/NewApiTest.php | 153 ++++++++++++++++++ tests/SentimentResultTest.php | 95 +++++++++++ 9 files changed, 626 insertions(+) create mode 100644 src/Exceptions/InvalidLexiconTermException.php create mode 100644 src/SentimentResult.php create mode 100644 tests/NewApiTest.php create mode 100644 tests/SentimentResultTest.php diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ba2f21..31fbe87 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,24 @@ This project follows [Semantic Versioning](https://semver.org/). verified on PHP 8.1–8.4 in CI. This release modernizes the codebase; it does not touch the scoring path. See `MIGRATION.md`. +### Added — new API + +- `Analyzer::analyze(string): SentimentResult` — an immutable result object with + `compound()`, `positive()`, `negative()`, `neutral()`, `label()`, + `isPositive()`/`isNegative()`/`isNeutral()` and `toArray()`. +- `Analyzer::analyzeMany(iterable): SentimentResult[]` — preserves input keys. +- `Analyzer::withLexicon(array): static` — immutable; returns a new analyzer. + Stricter than `updateLexicon()`: rejects multi-word terms and non-numeric + values instead of silently coercing or ignoring them. +- `SentimentResult::POSITIVE_THRESHOLD` / `NEGATIVE_THRESHOLD` (±0.05, the VADER + convention) so callers can reclassify without hardcoding. + +`getSentiment()` is unaffected and returns the same array as always. Note that +`SentimentResult::toArray()` uses `positive`/`negative`/`neutral` where the +legacy array uses `pos`/`neg`/`neu` — see `MIGRATION.md`. + +`explain()` is not included; it is scheduled for 2.2. + ### Changed — BREAKING - **PHP 8.1+ is now required** (`^8.1`). Users on older runtimes stay on `1.x`, diff --git a/MIGRATION.md b/MIGRATION.md index 80e7a99..b71dca9 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -40,6 +40,50 @@ $analyzer->updateLexicon(['rubbish' => -1.5]); // same lowercasing, same coercio - The constructor keeps resolving lexicon paths relative to the package's `src/` directory. +## 2b. The new API (optional) + +Nothing below is required. `getSentiment()` keeps working exactly as before; the +new API is opt-in and layered over it, returning the same numbers. + +```php +$result = $analyzer->analyze('This update is really good!'); + +$result->compound(); // 0.6892 +$result->label(); // 'positive' +$result->isPositive(); // true +$result->toArray(); // ['positive' => …, 'negative' => …, 'neutral' => …, 'compound' => …, 'label' => …] +``` + +**`toArray()` keys differ from `getSentiment()` on purpose.** The legacy shape +(`neg`/`neu`/`pos`) is frozen and cannot be renamed; the new one spells the words +out. Do not mix them up — `SentimentResult` does not implement `ArrayAccess`, so +the two can never be swapped silently. + +Labels use the VADER convention, exposed as constants: +`compound >= 0.05` is positive, `<= -0.05` negative, neutral between. + +```php +$results = $analyzer->analyzeMany(['a' => 'great', 'b' => 'awful']); // keys preserved + +$custom = $analyzer->withLexicon(['slaps' => 2.2]); // returns a NEW analyzer +``` + +`withLexicon()` is **immutable** — assign the return value; the original is +unchanged. It is also stricter than the legacy `updateLexicon()`, which stays +lenient: + +| Input | `updateLexicon()` (legacy) | `withLexicon()` (new) | +|---|---|---| +| `['good' => 'abc']` | coerced to `0` | throws `InvalidLexiconTermException` | +| `['cut the mustard' => 3]` | silently does nothing | throws `InvalidLexiconTermException` | +| `['GOOD' => 1.5]` | lowercased | lowercased | + +Multi-word terms are rejected rather than routed into the idiom table, because +that matcher has known defects (`KNOWN-DIVERGENCES.md` §2) and would apply them +only in some positions. A clear error beats a feature that works sometimes. + +`explain()` is not in 2.0 — it is scheduled for 2.2. + ## 3. Accepted breaks ### 3.1 Internal methods are now private diff --git a/README.md b/README.md index 26860ba..a63ce7f 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ PHP Sentiment Analyzer is a lexicon and rule-based sentiment analysis tool that ## Contents * [Install](#install) +* [Modern API](#modern-api) * [Simple Usage](#simple-usage) * [Advanced Usage](#advanced-usage) * [Upgrading](#upgrading) @@ -45,6 +46,57 @@ Run the following to include this via Composer composer require davmixcool/php-sentiment-analyzer ``` +### Modern API + +Available from 2.0. Returns an immutable result object instead of a bare array. + +```php +use Sentiment\Analyzer; + +$analyzer = new Analyzer(); +$result = $analyzer->analyze('This update is really good!'); + +$result->compound(); // 0.5355 +$result->label(); // 'positive' +$result->isPositive(); // true +$result->positive(); // 0.463 +$result->toArray(); // ['positive' => 0.463, 'negative' => 0.0, 'neutral' => 0.537, 'compound' => 0.5355, 'label' => 'positive'] +``` + +Labels follow the VADER convention and are exposed as constants, so you can +reclassify without hardcoding: `compound >= 0.05` is positive, `<= -0.05` is +negative, and anything between is neutral. + +**Batch analysis** preserves your input keys, so results line up with their +source rows: + +```php +$results = $analyzer->analyzeMany([ + 'ticket-1' => 'This update is really good!', + 'ticket-2' => 'This product is terrible.', + 'ticket-3' => 'It works fine.', +]); + +$results['ticket-2']->label(); // 'negative' +$results['ticket-2']->compound(); // -0.4767 +``` + +**Custom lexicons** return a *new* analyzer — the original is untouched: + +```php +$slang = $analyzer->withLexicon([ + 'slaps' => 2.2, + 'mid' => -1.7, +]); + +$slang->analyze('that beat slaps')->compound(); // 0.4939 +$slang->analyze('the update is mid')->compound(); // -0.4019 +$analyzer->analyze('that beat slaps')->compound(); // 0.0 — unchanged +``` + +`withLexicon()` rejects multi-word terms and non-numeric values rather than +coercing them. The older `updateLexicon()` below stays lenient and unchanged. + ### Simple Usage ```php diff --git a/src/Analyzer.php b/src/Analyzer.php index f53f182..6abdab8 100644 --- a/src/Analyzer.php +++ b/src/Analyzer.php @@ -4,6 +4,7 @@ use Sentiment\Config\Config; use Sentiment\Exceptions\InvalidLexiconException; +use Sentiment\Exceptions\InvalidLexiconTermException; use Sentiment\Procedures\SentiText; /* @@ -223,6 +224,82 @@ public function getSentiment(string $text): array } + /** + * Analyze one piece of text. + * + * Delegates to getSentiment(), so scores are identical to the legacy method + * by construction — enforced across the whole characterization corpus by + * CharacterizationTest::testAnalyzeAgreesWithGetSentimentAcrossTheBaseline(). + */ + public function analyze(string $text): SentimentResult + { + return SentimentResult::fromScores($this->getSentiment($text)); + } + + /** + * Analyze many texts at once. + * + * Input keys are preserved so callers can correlate results with their + * source rows. + * + * @param iterable $texts + * @return array + */ + public function analyzeMany(iterable $texts): array + { + $results = []; + + foreach ($texts as $key => $text) { + $results[$key] = $this->analyze($text); + } + + return $results; + } + + /** + * Return a NEW analyzer with the given terms applied over the lexicon. + * + * Immutable: the receiver is untouched. Cloning rather than constructing + * avoids re-parsing ~11,000 lines of lexicon files, and PHP arrays are + * copy-on-write so the copy is cheap. + * + * Custom terms override defaults; across calls, last write wins. + * + * Unlike the legacy updateLexicon(), this rejects bad input instead of + * silently coercing it. + * + * @param array $terms + * @throws InvalidLexiconTermException + */ + public function withLexicon(array $terms): static + { + $clone = clone $this; + + foreach ($terms as $term => $valence) { + $term = (string) $term; + + if (preg_match('/\s/u', $term) === 1) { + throw InvalidLexiconTermException::multiWord($term); + } + + if (!is_numeric($valence)) { + throw InvalidLexiconTermException::nonNumeric($term, $valence); + } + + $clone->lexicon[strtolower($term)] = (float) $valence; + } + + return $clone; + } + + /** + * $current_sentitext is transient per-call state; a clone must not share it. + */ + public function __clone(): void + { + $this->current_sentitext = null; + } + /** @return array */ private function str_split_unicode(string $str): array { diff --git a/src/Exceptions/InvalidLexiconTermException.php b/src/Exceptions/InvalidLexiconTermException.php new file mode 100644 index 0000000..3f11f17 --- /dev/null +++ b/src/Exceptions/InvalidLexiconTermException.php @@ -0,0 +1,36 @@ +positive; + } + + public function negative(): float + { + return $this->negative; + } + + public function neutral(): float + { + return $this->neutral; + } + + public function compound(): float + { + return $this->compound; + } + + public function label(): string + { + if ($this->compound >= self::POSITIVE_THRESHOLD) { + return self::LABEL_POSITIVE; + } + + if ($this->compound <= self::NEGATIVE_THRESHOLD) { + return self::LABEL_NEGATIVE; + } + + return self::LABEL_NEUTRAL; + } + + public function isPositive(): bool + { + return $this->label() === self::LABEL_POSITIVE; + } + + public function isNegative(): bool + { + return $this->label() === self::LABEL_NEGATIVE; + } + + public function isNeutral(): bool + { + return $this->label() === self::LABEL_NEUTRAL; + } + + /** + * NOTE: these keys intentionally differ from getSentiment()'s legacy + * neg/neu/pos. The legacy shape is frozen and cannot be renamed; the new + * one should read clearly. See MIGRATION.md. + * + * @return array{positive: float, negative: float, neutral: float, compound: float, label: string} + */ + public function toArray(): array + { + return [ + 'positive' => $this->positive, + 'negative' => $this->negative, + 'neutral' => $this->neutral, + 'compound' => $this->compound, + 'label' => $this->label(), + ]; + } +} diff --git a/tests/CharacterizationTest.php b/tests/CharacterizationTest.php index 2344e62..92744aa 100644 --- a/tests/CharacterizationTest.php +++ b/tests/CharacterizationTest.php @@ -87,6 +87,42 @@ public function testScoresMatchBaseline(string $key, array $case): void ); } + /** + * The new API must never drift from the scoring contract. + * + * Rather than duplicating 355 fixture cases, assert that analyze() agrees + * with the pinned values everywhere. If analyze() ever stops delegating to + * getSentiment(), this fails against the same golden master. + */ + #[DataProvider('provideCases')] + public function testAnalyzeAgreesWithGetSentimentAcrossTheBaseline(string $key, array $case): void + { + if (isset($case['lexicon'])) { + $analyzer = new Analyzer(); + $analyzer->updateLexicon($case['lexicon']); + } else { + $analyzer = self::sharedAnalyzer(); + } + + $result = $analyzer->analyze($case['text']); + + $actual = [ + 'neg' => sprintf('%.3f', $result->negative()), + 'neu' => sprintf('%.3f', $result->neutral()), + 'pos' => sprintf('%.3f', $result->positive()), + 'compound' => sprintf('%.4f', $result->compound()), + ]; + + $expected = [ + 'neg' => $case['neg'], + 'neu' => $case['neu'], + 'pos' => $case['pos'], + 'compound' => $case['compound'], + ]; + + $this->assertSame($expected, $actual, sprintf('analyze() drifted on "%s"', $key)); + } + public function testBaselineCoversEveryRuleTableEntry(): void { $keys = array_keys(self::provideCases()); diff --git a/tests/NewApiTest.php b/tests/NewApiTest.php new file mode 100644 index 0000000..5c7aa9f --- /dev/null +++ b/tests/NewApiTest.php @@ -0,0 +1,153 @@ +assertInstanceOf(SentimentResult::class, self::analyzer()->analyze('this is good')); + } + + public function testAnalyzeMatchesGetSentiment(): void + { + $analyzer = self::analyzer(); + $legacy = $analyzer->getSentiment('this is good'); + $result = $analyzer->analyze('this is good'); + + $this->assertSame($legacy['compound'], $result->compound()); + $this->assertSame($legacy['pos'], $result->positive()); + } + + public function testAnalyzeManyPreservesStringKeys(): void + { + $results = self::analyzer()->analyzeMany([ + 'first' => 'This is amazing!', + 'second' => 'This product is terrible.', + ]); + + $this->assertSame(['first', 'second'], array_keys($results)); + $this->assertTrue($results['first']->isPositive()); + $this->assertTrue($results['second']->isNegative()); + } + + public function testAnalyzeManyPreservesSparseIntegerKeys(): void + { + $results = self::analyzer()->analyzeMany([5 => 'good', 9 => 'terrible']); + + $this->assertSame([5, 9], array_keys($results)); + } + + public function testAnalyzeManyAcceptsATraversable(): void + { + $results = self::analyzer()->analyzeMany(new \ArrayIterator(['a' => 'good'])); + + $this->assertSame(['a'], array_keys($results)); + } + + public function testAnalyzeManyOnEmptyInput(): void + { + $this->assertSame([], self::analyzer()->analyzeMany([])); + } + + public function testWithLexiconIsImmutable(): void + { + $original = new Analyzer(); + $before = $original->getSentiment('it was slaps')['compound']; + + $modified = $original->withLexicon(['slaps' => 2.2]); + + $this->assertNotSame($original, $modified, 'withLexicon() must return a new instance'); + $this->assertSame( + $before, + $original->getSentiment('it was slaps')['compound'], + 'the receiver must be unchanged' + ); + $this->assertGreaterThan($before, $modified->getSentiment('it was slaps')['compound']); + } + + public function testWithLexiconLastWriteWins(): void + { + $analyzer = (new Analyzer()) + ->withLexicon(['slaps' => 2.2]) + ->withLexicon(['slaps' => -2.2]); + + $this->assertLessThan(0, $analyzer->analyze('it was slaps')->compound()); + } + + public function testWithLexiconLowercasesKeys(): void + { + $upper = (new Analyzer())->withLexicon(['SNAPPY' => 1.8]); + $lower = (new Analyzer())->withLexicon(['snappy' => 1.8]); + + $this->assertSame( + $lower->analyze('it was snappy')->compound(), + $upper->analyze('it was snappy')->compound() + ); + } + + public function testWithLexiconRejectsMultiWordTerms(): void + { + // Routing these into the idiom table would work only in some positions, + // because that matcher has known defects. Fail loudly instead. + $this->expectException(InvalidLexiconTermException::class); + $this->expectExceptionMessageMatches('/Multi-word term/'); + + (new Analyzer())->withLexicon(['cut the mustard' => 3.0]); + } + + public function testWithLexiconRejectsNonNumericValues(): void + { + $this->expectException(InvalidLexiconTermException::class); + $this->expectExceptionMessageMatches('/must be numeric/'); + + (new Analyzer())->withLexicon(['clunky' => 'very bad']); + } + + public function testWithLexiconAcceptsNumericStrings(): void + { + // README examples pass strings like '-1.5'; these stay valid. + $analyzer = (new Analyzer())->withLexicon(['rubbish' => '-1.5']); + + $this->assertLessThan(0, $analyzer->analyze('it was rubbish')->compound()); + } + + public function testLegacyUpdateLexiconStaysLenientWhileWithLexiconIsStrict(): void + { + // The BC contract freezes updateLexicon()'s coercion. The new API is + // allowed to be strict; the old one is not allowed to change. + $legacy = new Analyzer(); + $legacy->updateLexicon(['clunky' => 'very bad']); + $this->assertSame(0.0, $legacy->getSentiment('it was clunky')['compound']); + + $this->expectException(InvalidLexiconTermException::class); + (new Analyzer())->withLexicon(['clunky' => 'very bad']); + } + + public function testCloneDoesNotShareTransientState(): void + { + $original = new Analyzer(); + $original->getSentiment('THIS IS GREAT'); + + $clone = $original->withLexicon(['slaps' => 2.2]); + + // A shared SentiText would leak the previous call's caps-differential + // flag into the clone's first scoring run. + $this->assertSame( + (new Analyzer())->withLexicon(['slaps' => 2.2])->getSentiment('it was slaps'), + $clone->getSentiment('it was slaps') + ); + } +} diff --git a/tests/SentimentResultTest.php b/tests/SentimentResultTest.php new file mode 100644 index 0000000..ea5b802 --- /dev/null +++ b/tests/SentimentResultTest.php @@ -0,0 +1,95 @@ +assertSame(0.746, $result->positive()); + $this->assertSame(0.0, $result->negative()); + $this->assertSame(0.254, $result->neutral()); + $this->assertSame(0.8316, $result->compound()); + } + + public function testFromScoresMapsLegacyKeys(): void + { + // The legacy shape uses neg/neu/pos; mixing these up would be silent + // and catastrophic, so pin the mapping explicitly. + $result = SentimentResult::fromScores([ + 'neg' => 0.1, + 'neu' => 0.2, + 'pos' => 0.7, + 'compound' => 0.5, + ]); + + $this->assertSame(0.1, $result->negative()); + $this->assertSame(0.2, $result->neutral()); + $this->assertSame(0.7, $result->positive()); + $this->assertSame(0.5, $result->compound()); + } + + public static function provideLabels(): array + { + return [ + 'well above threshold' => [0.8316, 'positive'], + 'exactly at positive threshold' => [0.05, 'positive'], + 'just below positive threshold' => [0.0499, 'neutral'], + 'zero' => [0.0, 'neutral'], + 'just above negative threshold' => [-0.0499, 'neutral'], + 'exactly at negative threshold' => [-0.05, 'negative'], + 'well below threshold' => [-0.5423, 'negative'], + ]; + } + + #[DataProvider('provideLabels')] + public function testLabelBoundaries(float $compound, string $expected): void + { + $this->assertSame($expected, self::make($compound)->label()); + } + + public function testThresholdsAreTheVaderConventionAndPublic(): void + { + // Public so callers can reclassify without hardcoding. Changing these + // silently reclassifies every result, so pin them. + $this->assertSame(0.05, SentimentResult::POSITIVE_THRESHOLD); + $this->assertSame(-0.05, SentimentResult::NEGATIVE_THRESHOLD); + } + + public function testPredicatesAreMutuallyExclusive(): void + { + foreach ([0.8, 0.0, -0.8] as $compound) { + $result = self::make($compound); + + $true = array_filter([ + $result->isPositive(), + $result->isNeutral(), + $result->isNegative(), + ]); + + $this->assertCount(1, $true, "Exactly one predicate must hold for {$compound}"); + } + } + + public function testToArrayUsesTheNewKeyNames(): void + { + // Deliberately different from getSentiment()'s frozen neg/neu/pos. + $result = new SentimentResult(0.76, 0.0, 0.24, 0.81); + + $this->assertSame( + ['positive' => 0.76, 'negative' => 0.0, 'neutral' => 0.24, 'compound' => 0.81, 'label' => 'positive'], + $result->toArray() + ); + } +} From 9048addb446b7f6d8e1dda5e9c30999f415e3b4c Mon Sep 17 00:00:00 2001 From: David Oti Date: Wed, 19 Aug 2026 11:00:54 +0100 Subject: [PATCH 4/7] Refactor documentation in Known Divergences and tests to clarify backward compatibility and migration details --- KNOWN-DIVERGENCES.md | 14 +++++++------- tests/ApiContractTest.php | 10 ++++++---- tests/CharacterizationTest.php | 5 ++--- tools/generate-baseline.php | 3 ++- 4 files changed, 17 insertions(+), 15 deletions(-) diff --git a/KNOWN-DIVERGENCES.md b/KNOWN-DIVERGENCES.md index 20bdc57..5cd3fe3 100644 --- a/KNOWN-DIVERGENCES.md +++ b/KNOWN-DIVERGENCES.md @@ -3,10 +3,9 @@ Behaviour that is **pinned in `tests/fixtures/baseline.json` because it is what the code currently does — not because it is correct.** -v2.0 guarantees byte-identical scores with v1 (see the Scoring Parity section of -the v2 PRD). Everything still listed as outstanding below is therefore -reproduced exactly in v2.0 and fixed in a later release, each with a -`CHANGELOG.md` entry. +v2.0 guarantees byte-identical scores with v1. Everything still listed as +outstanding below is therefore reproduced exactly in v2.0 and fixed in a later +release, each with a `CHANGELOG.md` entry. Items marked FIXED were corrected deliberately, with their pinned cases re-based in the same commit and the movement documented here. @@ -79,9 +78,10 @@ which only agrees trivially because its table value is zero). Corpus sections: `idiom/*`, `idiom_sentence/*`. -**Note for v2:** this interacts with the PRD's custom-lexicon design. Multi-word -keys passed to `withLexicon()` land in the term lexicon, not the idiom table, so -they cannot work until the idiom matcher does. +**Note for v2:** this constrains custom lexicons. Multi-word keys passed to +`withLexicon()` would land in the term lexicon, not the idiom table, so they +cannot work until the idiom matcher does — which is why `withLexicon()` rejects +them outright rather than accepting them and doing nothing. --- diff --git a/tests/ApiContractTest.php b/tests/ApiContractTest.php index cd7f2a9..c1e28e3 100644 --- a/tests/ApiContractTest.php +++ b/tests/ApiContractTest.php @@ -8,7 +8,7 @@ use Sentiment\Exceptions\InvalidLexiconException; /** - * Pins the frozen backward-compatibility surface from PRD §3. + * Pins the frozen backward-compatibility surface documented in MIGRATION.md. * * These are the guarantees v2 must not break. Unlike CharacterizationTest, * which pins numbers, this pins SHAPE: return keys, mutation semantics, and @@ -27,8 +27,9 @@ public function testGetSentimentReturnsFrozenKeysInOrder(): void public function testGetSentimentReturnsPlainArrayNotAnObject(): void { - // PRD §3: getSentiment() must NOT return SentimentResult, and - // SentimentResult must NOT implement ArrayAccess. Assert the DECLARED + // getSentiment() must NOT return SentimentResult, and SentimentResult + // must NOT implement ArrayAccess — bridging the two shapes is where + // subtle breakage hides. Assert the DECLARED // return type rather than the runtime value — that is what callers // and static analysis actually depend on, and a change to an object // type would be caught here even if it were array-like at runtime. @@ -150,7 +151,8 @@ public function testMissingEmojiLexiconThrows(): void public function testNoRuntimeCodePathPerformsNetworkIo(): void { - // PRD: "Runtime inference must never require a network connection." + // Inference must never require a network connection: the package is + // local and deterministic by design. $forbidden = [ 'file_get_contents(http', 'curl_init', 'curl_exec', 'fsockopen', 'stream_socket_client', 'fopen(http', 'http_get', 'socket_create', diff --git a/tests/CharacterizationTest.php b/tests/CharacterizationTest.php index 92744aa..7731894 100644 --- a/tests/CharacterizationTest.php +++ b/tests/CharacterizationTest.php @@ -9,9 +9,8 @@ /** * Golden-master suite pinning the behaviour of the CURRENT implementation. * - * Milestone 0 of the v2 PRD: this must run green against unmodified v1 before - * any refactor begins, and must stay green through v2.0, which guarantees - * byte-identical scores. + * This must run green against unmodified v1 before any refactor begins, and + * must stay green through v2.0, which guarantees byte-identical scores. * * A failure here means a score moved. That is a regression unless it is an * intentional, changelogged scoring change — and those are out of scope for diff --git a/tools/generate-baseline.php b/tools/generate-baseline.php index ed1e0c6..a58ee3d 100644 --- a/tools/generate-baseline.php +++ b/tools/generate-baseline.php @@ -282,7 +282,8 @@ function readEmoji(): array $corpus[$key] = ['text' => $text, 'lexicon' => $readmeLexicon]; } -// updateLexicon() semantics frozen by the BC contract (PRD §3). +// updateLexicon() semantics are frozen by the backward-compatibility +// contract; see MIGRATION.md. $corpus['custom_lexicon/new_term'] = ['text' => 'it was slaps', 'lexicon' => ['slaps' => 2.2]]; $corpus['custom_lexicon/override'] = ['text' => 'it was good', 'lexicon' => ['good' => -3.0]]; $corpus['custom_lexicon/uppercase_key'] = ['text' => 'it was snappy', 'lexicon' => ['SNAPPY' => 1.8]]; From 396535cb0f45849cf244a21eb0134e9096d09f37 Mon Sep 17 00:00:00 2001 From: David Oti Date: Wed, 19 Aug 2026 13:15:08 +0100 Subject: [PATCH 5/7] Add NOTICE.md and update README.md for third-party lexicon attribution; fix license link in README --- CHANGELOG.md | 10 +++++++ NOTICE.md | 65 ++++++++++++++++++++++++++++++++++++++++++ README.md | 10 +++++-- src/Lexicons/README.md | 48 +++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 2 deletions(-) create mode 100644 NOTICE.md create mode 100644 src/Lexicons/README.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 31fbe87..9c7fa23 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,16 @@ legacy array uses `pos`/`neg`/`neu` — see `MIGRATION.md`. `Config`. - PHPStan at level 5, wired into CI. - `MIGRATION.md`. +- `NOTICE.md` and `src/Lexicons/README.md` — attribution and full MIT license + text for the bundled VADER sentiment and emoji lexicons, which are + third-party data from [cjhutto/vaderSentiment](https://github.com/cjhutto/vaderSentiment) + (Copyright (c) 2016 C.J. Hutto). Both ship in the release tarball, as the + license requires. The data itself is unchanged. + +### Fixed — documentation + +- The README's MIT license links pointed at `/blob/master/LICENSE`, which 404s; + the file is `LICENCE.txt`. ## [1.3.0] - 2026-08-19 diff --git a/NOTICE.md b/NOTICE.md new file mode 100644 index 0000000..507f0ff --- /dev/null +++ b/NOTICE.md @@ -0,0 +1,65 @@ +# Third-Party Notices + +This package redistributes data files that are **not** its own work. They are +included in every release so that sentiment analysis works offline, with no +network access at runtime. + +The package's own source code is licensed separately under `LICENCE.txt`. + +--- + +## VADER Sentiment Lexicon and Emoji Lexicon + +**Files** + +- `src/Lexicons/vader_sentiment_lexicon.txt` +- `src/Lexicons/emoji_utf8_lexicon.txt` + +**Source:** [cjhutto/vaderSentiment](https://github.com/cjhutto/vaderSentiment) + +**Modifications:** none of substance. The data is upstream's, with no term +added, removed, or revalued. `emoji_utf8_lexicon.txt` is byte-identical. +`vader_sentiment_lexicon.txt` differs from upstream only by a UTF-8 byte-order +mark on its first line and the absence of a trailing newline — both accidents of +copying, not edits to the data. The byte-order mark has a known side effect, and +is documented in `KNOWN-DIVERGENCES.md`. + +**License:** MIT, reproduced in full below as required. + +``` +The MIT License (MIT) + +Copyright (c) 2016 C.J. Hutto + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE.``` + +**Citation** + +> Hutto, C.J. & Gilbert, E.E. (2014). VADER: A Parsimonious Rule-based Model for +> Sentiment Analysis of Social Media Text. Eighth International Conference on +> Weblogs and Social Media (ICWSM-14). Ann Arbor, MI, June 2014. + +--- + +## This package + +Everything outside `src/Lexicons/` is the work of this package's authors and is +licensed under the MIT License in `LICENCE.txt`. The two licenses are separate: +`LICENCE.txt` does not grant rights to the lexicon data, and the notice above +does not cover this package's code. diff --git a/README.md b/README.md index a63ce7f..3a31513 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ PHP Sentiment Analyzer is a lexicon and rule-based sentiment analysis tool that is used to understand sentiments in a sentence using VADER \(Valence Aware Dictionary and sentiment Reasoner\). -[![GitHub license](https://img.shields.io/github/license/davmixcool/php-sentiment-analyzer.svg)](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/LICENSE) [![GitHub issues](https://img.shields.io/github/issues/davmixcool/php-sentiment-analyzer.svg)](https://github.com/davmixcool/php-sentiment-analyzer/issues) [![Stable](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/v/stable.svg)](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/v/stable.svg) [![Download](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/d/total.svg)](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/d/total.svg) [![Twitter](https://img.shields.io/twitter/url/https/github.com/davmixcool/php-sentiment-analyzer.svg?style=social)](https://twitter.com/intent/tweet?text=Wow:&url=https%3A%2F%2Fgithub.com%2Fdavmixcool%2Fphp-sentiment-analyzer) +[![GitHub license](https://img.shields.io/github/license/davmixcool/php-sentiment-analyzer.svg)](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/LICENCE.txt) [![GitHub issues](https://img.shields.io/github/issues/davmixcool/php-sentiment-analyzer.svg)](https://github.com/davmixcool/php-sentiment-analyzer/issues) [![Stable](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/v/stable.svg)](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/v/stable.svg) [![Download](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/d/total.svg)](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/d/total.svg) [![Twitter](https://img.shields.io/twitter/url/https/github.com/davmixcool/php-sentiment-analyzer.svg?style=social)](https://twitter.com/intent/tweet?text=Wow:&url=https%3A%2F%2Fgithub.com%2Fdavmixcool%2Fphp-sentiment-analyzer) ## Features @@ -210,7 +210,13 @@ affected text after upgrading. Full details in the ### License -This package is licensed under the [MIT license](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/LICENSE). +The package's source code is licensed under the +[MIT license](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/LICENCE.txt). + +The bundled sentiment and emoji lexicons in `src/Lexicons/` are **third-party +data**, redistributed from [cjhutto/vaderSentiment](https://github.com/cjhutto/vaderSentiment) +under its own MIT license (Copyright (c) 2016 C.J. Hutto). Full attribution and +license text are in [NOTICE.md](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/NOTICE.md). ### Reference diff --git a/src/Lexicons/README.md b/src/Lexicons/README.md new file mode 100644 index 0000000..e724df8 --- /dev/null +++ b/src/Lexicons/README.md @@ -0,0 +1,48 @@ +# Lexicon data + +These files are **third-party data, not this package's work**. See +[`NOTICE.md`](../../NOTICE.md) in the repository root for the full license text +and attribution. + +| File | Lines | Source | +|---|---|---| +| `vader_sentiment_lexicon.txt` | 7,519 | [cjhutto/vaderSentiment](https://github.com/cjhutto/vaderSentiment) (MIT) | +| `emoji_utf8_lexicon.txt` | 3,569 | [cjhutto/vaderSentiment](https://github.com/cjhutto/vaderSentiment) (MIT) | + +Both are tab-separated. The sentiment lexicon is +`termvalencestandard deviationraw ratings`; only the first two +columns are read. The emoji lexicon is `emojidescription`. + +## Do not add comments or headers to these files + +The parser (`Analyzer::make_lex_dict()`) splits **every** line on tabs and reads +the first two fields. A comment line has no tab, so it produces +`Warning: Undefined array key 1` — and the test suite runs with +`failOnWarning="true"`, so it will fail the build. Provenance belongs in this +README, not in the data. + +## Do not "clean up" the byte-order mark + +`vader_sentiment_lexicon.txt` begins with a UTF-8 BOM. It is not harmless: it +makes the first entry parse as `\xEF\xBB\xBF$:` rather than `$:`, so that one +emoticon is unreachable. + +Removing it would make the term match and **change sentiment scores** for any +text containing `$:`. That is a scoring change, and this package guarantees +byte-identical scores within a release line. It is catalogued in +`KNOWN-DIVERGENCES.md` and will be fixed in a release that says so. + +`.gitattributes` marks these files `-text` to keep any future line-ending +normalisation from rewriting them, which would shift every score in the package. + +## Changing the data + +Don't edit these files to customise sentiment. Use the runtime API instead: + +```php +$analyzer = $analyzer->withLexicon(['slaps' => 2.2]); // 2.0+ +$analyzer->updateLexicon(['slaps' => 2.2]); // legacy, all versions +``` + +If the data genuinely must change, regenerate the characterization baseline +(`composer baseline`) and review every score that moves. From 93f2bb0f14415038438bf81f127c4818f05d894b Mon Sep 17 00:00:00 2001 From: David Oti Date: Wed, 19 Aug 2026 14:26:32 +0100 Subject: [PATCH 6/7] Update README.md to enhance badge visibility and remove stargazers and forkers sections --- README.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/README.md b/README.md index 3a31513..2e11316 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ PHP Sentiment Analyzer is a lexicon and rule-based sentiment analysis tool that is used to understand sentiments in a sentence using VADER \(Valence Aware Dictionary and sentiment Reasoner\). -[![GitHub license](https://img.shields.io/github/license/davmixcool/php-sentiment-analyzer.svg)](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/LICENCE.txt) [![GitHub issues](https://img.shields.io/github/issues/davmixcool/php-sentiment-analyzer.svg)](https://github.com/davmixcool/php-sentiment-analyzer/issues) [![Stable](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/v/stable.svg)](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/v/stable.svg) [![Download](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/d/total.svg)](https://poser.pugx.org/davmixcool/php-sentiment-analyzer/d/total.svg) [![Twitter](https://img.shields.io/twitter/url/https/github.com/davmixcool/php-sentiment-analyzer.svg?style=social)](https://twitter.com/intent/tweet?text=Wow:&url=https%3A%2F%2Fgithub.com%2Fdavmixcool%2Fphp-sentiment-analyzer) +[![CI](https://img.shields.io/github/actions/workflow/status/davmixcool/php-sentiment-analyzer/ci.yml?branch=2.x&label=CI)](https://github.com/davmixcool/php-sentiment-analyzer/actions/workflows/ci.yml) [![Latest Version](https://img.shields.io/packagist/v/davmixcool/php-sentiment-analyzer?label=latest)](https://packagist.org/packages/davmixcool/php-sentiment-analyzer) [![PHP Version](https://img.shields.io/packagist/php-v/davmixcool/php-sentiment-analyzer/2.x-dev?label=php)](https://packagist.org/packages/davmixcool/php-sentiment-analyzer) [![Total Downloads](https://img.shields.io/packagist/dt/davmixcool/php-sentiment-analyzer)](https://packagist.org/packages/davmixcool/php-sentiment-analyzer) [![License](https://img.shields.io/packagist/l/davmixcool/php-sentiment-analyzer)](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/LICENCE.txt) [![Stars](https://img.shields.io/github/stars/davmixcool/php-sentiment-analyzer)](https://github.com/davmixcool/php-sentiment-analyzer/stargazers) [![Forks](https://img.shields.io/github/forks/davmixcool/php-sentiment-analyzer)](https://github.com/davmixcool/php-sentiment-analyzer/network/members) ## Features @@ -25,8 +25,6 @@ PHP Sentiment Analyzer is a lexicon and rule-based sentiment analysis tool that * [Simple Usage](#simple-usage) * [Advanced Usage](#advanced-usage) * [Upgrading](#upgrading) -* [Stargazers](#stargazers) -* [Forkers](#forkers) * [License](#license) * [Reference](#reference) @@ -200,14 +198,6 @@ If you store sentiment scores or compare them against thresholds, re-score any affected text after upgrading. Full details in the [changelog](https://github.com/davmixcool/php-sentiment-analyzer/blob/master/CHANGELOG.md). -### Stargazers - -[![Stargazers repo roster for @davmixcool/php-sentiment-analyzer](https://reporoster.com/stars/davmixcool/php-sentiment-analyzer)](https://github.com/davmixcool/php-sentiment-analyzer/stargazers) - -### Forkers - -[![Forkers repo roster for @davmixcool/php-sentiment-analyzer](https://reporoster.com/forks/davmixcool/php-sentiment-analyzer)](https://github.com/davmixcool/php-sentiment-analyzer/network/members) - ### License The package's source code is licensed under the From 4d6cd75f369990adead08f87f82d3010eb377c60 Mon Sep 17 00:00:00 2001 From: David Oti Date: Wed, 19 Aug 2026 14:32:19 +0100 Subject: [PATCH 7/7] Update CHANGELOG.md for version 2.0.0 release and add documentation for version 1.3.1 --- CHANGELOG.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c7fa23..9cd2e56 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ All notable changes to this project are documented here. This project follows [Semantic Versioning](https://semver.org/). -## [2.0.0] - unreleased +## [2.0.0] - 2026-08-19 ### Scores are unchanged @@ -65,6 +65,27 @@ legacy array uses `pos`/`neg`/`neu` — see `MIGRATION.md`. ### Fixed — documentation +- The README's MIT license links pointed at `/blob/master/LICENSE`, which 404s; + the file is `LICENCE.txt`. + +## [1.3.1] - 2026-08-19 + +### Scores are unchanged + +Documentation only. No source file was modified, and every pinned score in the +characterization suite is byte-identical to 1.3.0. **There is no need to +re-score stored text for this release.** + +### Added + +- `NOTICE.md` and `src/Lexicons/README.md` — attribution and the full MIT + license text for the bundled VADER sentiment and emoji lexicons, which are + third-party data from [cjhutto/vaderSentiment](https://github.com/cjhutto/vaderSentiment) + (Copyright (c) 2016 C.J. Hutto). Both ship in the release tarball, as the + license requires. The lexicon data itself is unchanged. + +### Fixed + - The README's MIT license links pointed at `/blob/master/LICENSE`, which 404s; the file is `LICENCE.txt`.