General Answer
We thank the reviewers for their thorough and helpful feedback.
**Main Contribution of the paper:**
Our paper is fundamentally theoretical, focusing on providing the first ever general convergence proof for approaches based on the Tsetlin Automaton. This proof not only demonstrates convergence for automaton-based methods but also lays the groundwork for potentially establishing a general convergence proof for the original Tsetlin Machine.
It is important to clarify that our convergence proof specifically pertains to the Tsetlin Automaton family, not the original Tsetlin Machine model itself. Currently, a comprehensive proof for the original Tsetlin Machine remains elusive, primarily due to the complex interplay between literals. Reviewer Q3AU highlighted a possible point of confusion, which we address here: Tsetlin Automatons [Tsetlin (1961)], which are fundamental components of both PCL and the original Tsetlin Machine (as detailed in Section 2), should not be mistaken for the Tsetlin Machine approach itself [Granmo (2018)].
**Usefulness of the contribution and practical implications:**
Establishing convergence in a machine learning model is essential because it ensures the model's reliability and stability. Specifically, in our context, proving convergence means that PCL will eventually learn an accurate hypothesis, which translates to an exact or nearly exact solution for practical problems.
**Experimental evaluation:**
Many reviewers noted that our paper lacks comprehensive experimental evaluation, particularly comparisons with state-of-the-art methods. We initially omitted detailed experimental results to maintain focus on our main contribution, the convergence proof. Nevertheless, in response to these requests, we have included some preliminary findings below and in the updated PDF. Currently, PCL functions as a binary classifier; evolving it into a multiclass classifier will necessitate changes in our implementation, which we are actively working on as time permits.
**PCL as classifier:**
In our study, we employed PCL as a classification tool, using its Disjunctive Normal Form (DNF) output, and tested it on binary iris and breast cancer datasets. We compared PCL's performance with that of the vanilla Tsetlin Machine (TM) and various established machine learning algorithms, with results detailed in Table 1. Notably, PCL achieved competitive results with just 10 clauses over 50 training epochs, setting clause probabilities uniformly in the range $[0.6, 0.8]$. For comparison, other methods used default settings from their sklearn implementations, while TM used $300$ clauses with specific settings $(s=2, T=10)$ over $100$ epochs. PCL, with the right $p_i$ settings, also attained a 96% accuracy rate on a binary MNIST dataset, distinguishing between 0s and 1s.
Additionally, we analyzed the convergence of PCL and TM using deterministic sample data, consisting of $n$ literals and a fixed target expression. We used this data to train and test the models, selecting samples from all possible combinations. Both PCL and TM were limited to two clauses. As shown in Table 2, PCL achieved 100\% accuracy aligning with our convergence proof. In contrast, TM reached 96.87\% and 93.50\% accuracy for $n=6$ and $n=10$, respectively. It is important to note that these accuracy figures, presented in Tables 1 and 2, are averages from $10$ independent runs.
| **Methods** | **Binary Iris** | **Breast Cancer** |
|------------------------|------------------|-------------------|
| Naive Bayes | 91.6 | 64.2 |
| Logistic Regression | 92.6 | 65.5 |
| 1-layer NN | 93.8 | 71.9 |
| SVM | 93.6 | 67.8 |
| DT | 94.7 | 70.6 |
| RF | 95.5 | 74.7 |
| KNN | 91.1 | 75.5 |
| TM (300 clauses) | 95.0 | 70.6 |
| PCL (10 clauses) | 94.2 | 69.5 |
*Table 1: Empirical performance comparison.*
| | **n=6** | **n=10** |
|-----------------------|-----------------|------------------|
| PCL (2 clauses) | 100 | 100 |
| TM (2 clauses) | 96.87 | 93.50 |
*Table 2: Noise-free example with deterministic target*
**[Tsetlin (1961)]** Michael Lvovitch Tsetlin. On Behaviour of Finite Automata in Random Medium. Avtomat. i
Telemekh, 22(10):1345–1354, 1961.
**[Granmo (2018)]** Ole-Christoffer Granmo. The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to
Optimal Pattern Recognition with Propositional Logic. arXiv preprint arXiv:1804.01508, 2018.