# Typestat

> This document goes over how TypeStart runs and generates fixes.

---

# Architecture

This document goes over how TypeStart runs and generates fixes.
Before reading this, you should read:

- TypeStat's [README.md](../README.md)
- Documentation on the types of [fixes](./Fixes.md)
- Recommended TypeStat [usage](./Usage.md)
- [automutate](https://github.com/automutate/automutate) and [automutate-tests](https://github.com/automutate/automutate)

## Runtime

When the `typestat` command is entered into the CLI, roughly the following happens in order:

1. [bin/typestat](../bin/typestat) calls to the [CLI](../src/cli/index.ts)
2. [Commander.js](https://github.com/tj/commander.js) parses the CLI's arguments
3. Settings are loaded from the `-c`/`--config` file
4. An Automutator provider is created for TypeStat with [`createTypeStatMutationsProvider`](../src/runtime/createTypeStatMutationsProvider.ts).

### Mutation Providers

There are three mutation providers that are run in order by [`createTypeStatMutationsProvider`](src/runtime/createTypeStatMutationsProvider.ts):

1. **Require renames**: changes to `import` and `require` statements from `files.renameExtensions`
2. **Core mutations**: changes to type annotations in provided files
3. **Files modified**: adds annotations to the top (`files.above`) and/or bottom (`files.below`) of mutated files if enabled

#### Require Renames

If any `require` to a file including the extension is stored as a variable, and `files.renameExtensions` is enabled,
that variable will be given a type equivalent to the extensionless equivalent.
This is done as a separate mutation provider before the core mutations to ensure these mutations are applied before core mutations.

> `import` declarations cannot be given different types, so they are ignored.

#### Core Mutations

Each round of mutations in the core mutation provider roughly:

1. Records the starting time
2. Creates a set of TypeScript language services
3. For each file to be visited:
   1. Retrieves file mutations for that file
   2. If more than 100 mutations have been collected, or mutations have been collected and it's been more than 10 seconds since the round started, stops the round

Subsequent rounds will pick up where the previous round left off.
For example, given files `a.ts`, `b.ts`, and `c.ts` in order,
if the first round runs on `a.ts` and `b.ts`, the second will start on `c.ts`.

Rounds stop after those thresholds to allow Automutate to write mutations regularly.
TypeStat crashing before a round is complete shouldn't lose all accumulated mutations.

Once TypeStat has visited each file, it will either:

- Stop if no file had mutations applied
- Restart _(and reload language services)_ if any file had mutations applied

#### File Mutations

For each file it visits, [`findMutationsInFile`](../src/runtime/findMutationsInFile.ts)
will attempt to apply [built-in file mutators](../src/mutators/builtIn/index.ts):

- [`fixIncompleteTypes`](../src/mutators/builtIn/fixIncompleteTypes/README.md)
- [`fixMissingProperties`](../src/mutators/builtIn/fixMissingProperties/README.md)
- [`fixNoImplicitAny`](../src/mutators/builtIn/fixNoImplicitAny/README.md)
- [`fixNoImplicitThis`](../src/mutators/builtIn/fixNoImplicitThis/README.md)
- [`fixStrictNonNullAssertions`](../src/mutators/builtIn/fixStrictNonNullAssertions/README.md)

Each fixer targets a general range of potential type improvements and contains a series of sub-fixers that target individual improvements.
For example, `fixIncompleteTypes` contains a `fixIncompleteParameterTypes` fixer that fills in incomplete types for parameters.

Within each round of applying mutations, TypeStat will stop looking at a file after each step if any mutations are found.
Adding mutations from one from can improve mutations from other forms, so reloading the file between rounds could reduce the number of later rounds.

> See [Fixes.md](./Fixes.md).

## Directory Structure

The following directories exist under `src/`:

### `logging`

Definition and default implementation of the logger for runtime.
It wraps `process.stderr` and `process.stdout` and is given to the runtime mutators as a member of `options`.

During tests, it'll be stubbed out.

### `mutators`

A file for each of the forms of mutation run in a round.
These are stored in the above fastest-first order in the exported [`builtInFileMutators`](../src/mutators/builtInFileMutators.ts) array.

### `mutations`

Called by the `mutators` to create mutations.
The `mutators` directory figures out which types should be modified, then `mutations` creates mutations to modify them.

We should note two common pieces of terminology used in this directory:

- **"Flags"** refers to type nodes for primitive types, such as `boolean`
- **"Types"** refers to rich (non-primitive) types, such as `MyClass`

### `options`

Parsing logic and TypeScript types for raw and parsed options.

[`loadOptions`](../src/options/loadOptions.ts) will `require` a `-c`/`--config` file from the path provided.
Options parsed from that file will be of type `RawTypeStatOptions`,
and will be filled out into `TypeStatOptions` via [`fillOutRawOptions`](../src/options/fillOutRawOptions.ts).

### `runtime`

Automutate hooks that launch and run the processes described above.
Think of this area as the coordinating force behind mutators.

### `services`

Creates wrappers around the [TypeScript Compiler API](https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API)
later used by mutators.
These include the [TypeScript Language Service API](https://github.com/Microsoft/TypeScript/wiki/Using-the-Language-Service-API).

Language services are passed to mutators in the form of a [`LanguageServices`](../src/services/language.ts) object.
They're recreated each mutations wave, as the underlying TypeScript source files generally change between wave.

### `shared`

Miscellaneous utilities used by other sections.

## Why Not [jscodeshift](https://github.com/facebook/jscodeshift)?

When I started on TypeStat, jscodeshift didn't yet support TypeScript.
Now it does.
Maybe this should be converted?

See [#20](https://github.com/JoshuaKGoldberg/TypeStat/issues/20).

---

# Cleanups

After TypeStat has applied all the fixes it can to files, there may still be some general cleanups that need to be applied.
Most commonly, any remaining TypeScript type errors may need to be suppressed with `// @ts-expect-error`.

Each classification of cleanups can be individually configured in your `typestat.json` file.
These all default to `false` but can be enabled by being set to `true`.

```json
{
	"cleanups": {
		"suppressTypeErrors": true
	}
}
```

## Cleaners

### `suppressTypeErrors`

Whether to add a `// @ts-expect-error` comment directive before each remaining type error.

See [suppressTypeErrors/README.md](../src/cleanups/builtin/suppressTypeErrors/README.md).

---

# Custom Mutators

TypeStat allows using custom mutators instead of the built-in mutators, similar to [custom ESLint rules](https://eslint.org/docs/developer-guide/working-with-rules).
Built-in mutators will be disabled if you provide any custom ones.

## Usage

Use the `-m`/`--mutators` CLI flag and/or `mutators` configuration setting to add `require`-style paths of mutators to add.

```json
{
	"mutators": ["my-mutator-module"]
}
```

These will be run in order of inclusion, starting with mutators specified on the CLI.

## Development

In order to create a custom mutator included by an added path, that added path must resolve from Node's `require` to a file that exports a `.mutator` function.

That `mutator` will receive a single `request` parameter of type [`FileMutationsRequest`](../src/mutators/fileMutator).
It should return an array of Automutate `Mutation` objects.

For example, if you run `typestat --add ./src/mutators/myMutator`, there should exist a `./src/mutators/myMutator.js` file _(or `./src/mutators/myMutator/index.js`)_:

```typescript
import { Mutation } from "automutate";
import { FileMutationsRequest } from "typestat";

export const mutator = (request: FileMutationsRequest): Mutation[] => {
	// TODO: Implement!
	return [];
};
```

Mutators must be compiled to JavaScript to be run.

For example, this mutator will add a `/* foo */` mutation at the beginning of each file it visits, if one doesn't yet exist in the file:

```js
const prefix = "/* foo */ ";

module.exports.fileMutator = (request) => {
	return request.sourceFile.getFullText().startsWith(prefix)
		? []
		: [
				{
					insertion: prefix,
					range: {
						begin: 0,
					},
					type: "text-insert",
				},
			];
};
```

---

# Development

After [forking the repo from GitHub](https://help.github.com/articles/fork-a-repo) and [installing pnpm](https://pnpm.io):

```shell
git clone https://github.com/<your-name-here>/TypeStat
cd TypeStat
pnpm
```

> This repository includes a list of suggested VS Code extensions.
> It's a good idea to use [VS Code](https://code.visualstudio.com) and accept its suggestion to install them, as they'll help with development.

## Building

Run [**tsup**](https://tsup.egoist.dev) locally to build source files from `src/` into output files in `lib/`:

```shell
pnpm build
```

Add `--watch` to run the builder in a watch mode that continuously cleans and recreates `lib/` as you save files:

```shell
pnpm build --watch
```

Once built, you can run TypeStat locally with `node bin/typestat.mjs`.

## Formatting

[Prettier](https://prettier.io) is used to format code.
It should be applied automatically when you save files in VS Code or make a Git commit.

To manually reformat all files, you can run:

```shell
pnpm format --write
```

## Linting

[ESLint](https://eslint.org) is used with with [typescript-eslint](https://typescript-eslint.io)) to lint JavaScript and TypeScript source files.
You can run it locally on the command-line:

```shell
pnpm run lint
```

ESLint can be run with `--fix` to auto-fix some lint rule complaints:

```shell
pnpm run lint --fix
```

Note that you'll likely need to run `pnpm build` before `pnpm lint` so that lint rules which check the file system can pick up on any built files.

## Testing

There are two kinds of tests:

- [Unit tests](#unit-tests)
- [Mutation tests](#mutation-tests)

### Unit Tests

[Vitest](https://vitest.dev) is used for tests.
You can run it locally on the command-line:

```shell
pnpm run test
```

Add the `--coverage` flag to compute test coverage and place reports in the `coverage/` directory:

```shell
pnpm run test --coverage
```

Note that [console-fail-test](https://github.com/JoshuaKGoldberg/console-fail-test) is enabled for all test runs.
Calls to `console.log`, `console.warn`, and other console methods will cause a test to fail.

#### Debugging Unit Tests

This repository includes a [VS Code launch configuration](https://code.visualstudio.com/docs/editor/debugging) for debugging unit tests.
To launch it, open a test file, then run _Debug Current Test File_ from the VS Code Debug panel (or press F5).

### Mutation Tests

Most TypeStat tests run TypeStat on checked-in files and are use snapshot testing for output.
These tests are located under `test/cases`.

[Vitest](https://vitest.dev) is also used for these tests.
To accept new snapshots, you can use [Vitest's snapshot updates](https://vitest.dev/guide/snapshot#updating-snapshots):

```shell
pnpm run test:mutation --update
```

#### Debugging Mutation Tests

VS Code tasks to debug test files is shipped that allows directly placing breakpoints in source TypeScript code.

- `Accept Current Mutation Test` runs with `-u`/`--update` on the test folder of a currently opened test file, such as an `original.ts` or `typestat.json` to update its snapshot.
- `Debug Current Test File` does not run with `-u`/`--update`, and thus treats any differences as test failures.

## Performance Debugging Tips

You can use the debugger in Chrome to debug TypeStat on the CLI.
Run it with `node --inspect` then visit `chrome://inspect` to use the browser debugger.

For example:

```shell
node --inspect typestat --config typestat.json
```

## Type Checking

You should be able to see suggestions from [TypeScript](https://typescriptlang.org) in your editor for all open files.

However, it can be useful to run the TypeScript command-line (`tsc`) to type check all files in `src/`:

```shell
pnpm tsc
```

Add `--watch` to keep the type checker running in a watch mode that updates the display as you save files:

```shell
pnpm tsc --watch
```

---

# Files

An optional set of configuration fields containing file-level changes to make outside of mutations.

```json
{
	"files": {
		"above": "/* Above file */",
		"below": "/* Below file */",
		"renameExtensions": true
	}
}
```

## `above`

```json
{
	"files": {
		"above": "/* Above file */"
	}
}
```

Comment to add above modified files, if any.
If provided, any modified file will have the text inserted as a new first line.

The default is `""`, for no action to take.
If a value is provided on the CLI, it will override a configuration file value (including `""`).

## `below`

```json
{
	"files": {
		"below": "/* Below file */"
	}
}
```

Comment to add below modified files, if any.
If provided, any modified file will have the text inserted as a new last line.

The default is `""`, for no action to take.
If a value is provided on the CLI, it will override a configuration file value (including `""`).

## `renameExtensions`

```json
{
	"files": {
		"renameExtensions": true
	}
}
```

Whether to convert `.js(x)` files to `.ts(x)`.
When this is enabled, any file with a JavaScript extension visited by TypeStat,
regardless of whether mutations are added, will be renamed to the equivalent TypeScript extension.

### Mapping Extensions

This field has four potential allowed configurations:

- `false` _(default)_: skip renaming file extensions
- `true`: auto-detect whether a file should be `.ts` or `.tsx`

  ```json
  {
  	"files": {
  		"renameExtensions": true
  	}
  }
  ```

- `"ts"`: always convert to `.ts`

  ```json
  {
  	"files": {
  		"renameExtensions": "ts"
  	}
  }
  ```

- `"tsx"`: always convert to `.tsx`

  ```json
  {
  	"files": {
  		"renameExtensions": "tsx"
  	}
  }
  ```

When auto-detection is enabled, a file will be converted to `.tsx` if either of the following is true:

- It `import`s or `require`s from the `"react"` module
- Its original file extension is `.jsx`

### Handling `require`s

While this option is enabled, if any `require` call to a file including the extension is stored as a variable,
that variable will be given a type equivalent to the extensionless equivalent.
For example:

```diff
- const sibling = require("./sibling.js");
+ const sibling: typeof import("./sibling") = require("./sibling.js");
```

This is necessary because TypeStat does not modify emitted JavaScript.
Removing extensions can sometimes cause unexpected behavior changes.

---

# Filters

TypeStat ships with built-in support for using [tsquery](https://github.com/phenomnomnominal/tsquery) to ignore sections of source files.
This is useful for...

- ...when sections of source files can be safely excluded from type coverage
- ...when you want to only touch up certain parts of source files

`filter` will _exclude_ any portions of source code that match them.
Sub-sections (child nodes) of those portions will not be visited.

For example, it's common in some architectures for classes to have `dispose()` methods _(mirroring C#'s `IDisposable`)_ where private members are set to `null`.
You can use filter to exclude these `null`s from type calculations.

## `filter`

```json
{
	"filter": ["MethodDeclaration[name.text=dispose]"]
}
```

---

# Fixes

TypeStat will apply mutations ("fixes") to files as it finds them.
These mutations are all purely additive and limited to the type system, meaning they will _not_ change your JavaScript output.

Each classification of fix can be individually configured in your `typestat.json` file.
These all default to `false` but can be enabled by being set to `true`.

```json
{
	"fixes": {
		"importExtensions": true,
		"incompleteTypes": true,
		"missingProperties": true,
		"noImplicitAny": true,
		"noImplicitThis": true,
		"noInferableTypes": true,
		"strictNonNullAssertions": true
	}
}
```

## Fixers

### `importExtensions`

Whether to add extensions to `export` and `import` declarations that refer to file paths without them.

See [fixImportExtensions/README.md](../src/mutators/builtIn/fixImportExtensions/README.md).

### `incompleteTypes`

Whether to augment type annotations that don't capture all values constructs can be set to.

See [fixIncompleteTypes/README.md](../src/mutators/builtIn/fixIncompleteTypes/README.md).

### `missingProperties`

Whether to apply TypeScript's fixer for missing properties on classes.

See [fixMissingProperties/README.md](../src/mutators/builtIn/fixMissingProperties/README.md).

### `noImplicitAny`

Whether to add type annotations to declarations that don't yet have them.

See [fixNoImplicitAny/README.md](../src/mutators/builtIn/fixNoImplicitAny/README.md).

### `noImplicitThis`

Whether to add `this` type annotations to functions that don't yet have them.

See [fixNoImplicitThis/README.md](../src/mutators/builtIn/fixNoImplicitThis/README.md).

### `noInferableTypes`

Whether to remove type annotations that don't change the meaning of code.

See [noInferableTypes/README.md](../src/mutators/builtIn/fixNoInferableTypes/README.md).

### `strictNonNullAssertions`

Whether to add missing non-null assertions.

See [fixStrictNonNullAssertions/README.md](../src/mutators/builtIn/fixStrictNonNullAssertions/README.md).

---

# Package

An optional set of configuration fields containing package-level changes to make outside of mutations.

```json
{
	"package": {
		"directory": "../MyRepo",
		"file": "./node/package.json",
		"missingTypes": "yarn"
	}
}
```

## `directory`

```json
{
	"package": {
		"directory": "../MyRepo"
	}
}
```

Base directory to resolve paths from.
All non-absolute paths within all settings except `-c`/`--config` will be resolved against this directory.

## `file`

```json
{
	"package": {
		"file": "./node/package.json"
	}
}
```

File path to a `package.json` to consider the project's package file.
If not provided, defaults to `./package.json`.

If `package.file` is relative, `package.directory` will be used as a root path to resolve from.

## `missingTypes`

```json
{
	"package": {
		"missingTypes": true
	}
}
```

Package manager to install missing types, if not `true` to auto-detect or `undefined` to not.
If this is provided, for any `require` or `import` to an absolute path that doesn't have its corresponding [`@types/`](https://github.com/DefinitelyTyped/DefinitelyTyped) package,
that package will be installed.

For example, if the following code exists in any file within the TypeScript project:

```javascript
import { array } from "lodash/array";
```

TypeStat will attempt to install `@types/lodash` unless it's already any form of dependency in the `package.file`,

### Package Manager Configuration

This field has four potential allowed configurations:

- `false` _(default)_: skip installing missing packages
- `true`: auto-detect whether to use Yarn _(if a `yarn.lock` exists)_ or npm _(default)_

  ```json
  {
  	"package": {
  		"missingTypes": true
  	}
  }
  ```

- `"npm"`: install using npm

  ```json
  {
  	"package": {
  		"missingTypes": "npm"
  	}
  }
  ```

- `"yarn"`: install using Yarn

  ```json
  {
  	"package": {
  		"missingTypes": "yarn"
  	}
  }
  ```

### Node types

`@types/node` will be installed in any of the following cases:

- `module =`, `module.exports =`, or `module.exports.*` = statement(s) exist
- [Built-in modules](https://www.npmjs.com/package/builtin-modules) are imported from
- A global [`process` object](https://nodejs.org/api/process.html#process_process) is referenced

---

# Types

## `strictNullChecks`

Whether to override the project's [`--strictNullChecks`](https://basarat.gitbooks.io/typescript/docs/options/strictNullChecks.html) setting.
If true, TypeStat will set `strictNullChecks` to `true` regardless of your `tsconfig.json`.

```json
{
	"types": {
		"strictNullChecks": true
	}
}
```

This interacts with fixers in a few ways:

- Type additions will now include `null` and/or `undefined`
- [Property Accesses](./Nodes.md#Strict%20Property%20Accesses) will have `!`s added as needed

---

# Usage

You'll need to tailor TypeStat's settings for your project.
It is **strongly recommended** to start with the `typestat` CLI tool to auto-generate a configuration file for you.

## Basic Usage

```shell
npx typestat
```

This will launch an interactive guide to setting up a `typestat.json` configuration file.
That file instructs subsequent runs to apply a series of "fixes" to your code.

```shell
npx typestat --config typestat.json
```

For example, the following `typestat.json` will add auto-fixes for missing type annotations to solve TypeScript's `noImplicitAny` complaints:

```json
{
	"fixes": {
		"noImplicitAny": true
	}
}
```

### Multi-Step Configurations

`typestat.json` can contain _either_ a single object describing fixes to make _or_ an array of those objects describing fixes to run in order.

For example, the following `typestat.json` will:

1. Add the above `noImplicitAny` fixes
2. Trim out any unnecessary types that TypeScript can infer from usage

```json
[
	{
		"fixes": {
			"noImplicitAny": true
		}
	},
	{
		"fixes": {
			"noInferableTypes": true
		}
	}
]
```

### Verbose Logging

Curious about how fixes are being suggested?
Run with a `--logfile` to get a detailed, verbose log of the exact fixes applied to each file.

```shell
npx typestat --logfile typestat.log
```

## More Examples

Use these examples as more granular references of how to perform targeted changes with TypeStat.

- [Converting Classes from JavaScript to TypeScript.md](./Usage/Converting%20Classes%20from%20JavaScript%20to%20TypeScript.md)
- [Enabling Strict Null Checks](./Usage/Enabling%20Strict%20Null%20Checks.md)