ditkrg/todo-to-issue-action
4
0
Fork 0
mirror of https://github.com/ditkrg/todo-to-issue-action.git synced 2026-01-22 22:06:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #168 from Christoph-Koschel/patch-3

Browse Source 
Add support for custom languages
This commit is contained in:
Alastair Mooney 2024-01-01 09:23:04 +00:00 committed by GitHub
commit 80f1c09e2b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 426 additions and 160 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

175
README.md
View File

@ -35,7 +35,8 @@ _Multiline_ TODOs are supported, with additional lines inserted into the issue b
print('Hello world!') print('Hello world!')
``` ```
As per the [Google Style Guide](https://google.github.io/styleguide/cppguide.html#TODO_Comments), you can provide a _reference_ after the TODO identifier. This will be included in the issue title for searchability. As per the [Google Style Guide](https://google.github.io/styleguide/cppguide.html#TODO_Comments), you can provide a
_reference_ after the TODO identifier. This will be included in the issue title for searchability.
```python ```python
def hello_world(): def hello_world():
@ -104,10 +105,13 @@ With some additional setup, you can assign the created issues a status (column)
By default, the action cannot access your projects. To enable it, you must: By default, the action cannot access your projects. To enable it, you must:
* [Create a Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token), * [Create a Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token),
* [Create an encrypted secret in your repo settings](https://docs.github.com/en/actions/reference/encrypted-secrets#creating-encrypted-secrets-for-a-repository), with the value set to the Personal Access Token, * [Create an encrypted secret in your repo settings](https://docs.github.com/en/actions/reference/encrypted-secrets#creating-encrypted-secrets-for-a-repository),
* Assign the secret in the workflow file like `PROJECTS_SECRET: ${{ secrets.PROJECTS_SECRET }}`. _Do not enter the raw secret_. with the value set to the Personal Access Token,
* Assign the secret in the workflow file like `PROJECTS_SECRET: ${{ secrets.PROJECTS_SECRET }}`. _Do not enter the raw
secret_.
Projects are identified by their `full project name and issue status` (column) reference with the `<user or org name>/project name/status name` syntax. Projects are identified by their `full project name and issue status` (column) reference with
the `<user or org name>/project name/status name` syntax.
* To assign to a _user project_, use the `user projects:` option. * To assign to a _user project_, use the `user projects:` option.
* To assign to an _organisation project_, use `org projects:` option. * To assign to an _organisation project_, use `org projects:` option.
@ -121,9 +125,11 @@ Projects are identified by their `full project name and issue status` (column) r
print('Hello world!') print('Hello world!')
``` ```
You can assign issues to multiple projects separating them with commas, i.e. `user projects: alstr/Test User Project 1/To Do, alstr/Test User Project 2/Tasks`. You can assign issues to multiple projects separating them with commas,
i.e. `user projects: alstr/Test User Project 1/To Do, alstr/Test User Project 2/Tasks`.
You can also specify `default projects` in the same way by defining `USER_PROJECTS` or `ORG_PROJECTS` in your workflow file. You can also specify `default projects` in the same way by defining `USER_PROJECTS` or `ORG_PROJECTS` in your workflow
file.
These will be applied automatically to every issue, but will be overrode by any specified within the TODO. These will be applied automatically to every issue, but will be overrode by any specified within the TODO.
## Supported Languages ## Supported Languages
@ -180,9 +186,11 @@ These will be applied automatically to every issue, but will be overrode by any
New languages can easily be added to the `syntax.json` file, used by the action to identify TODO comments. New languages can easily be added to the `syntax.json` file, used by the action to identify TODO comments.
When adding languages, follow the structure of existing entries, and use the language name defined by GitHub in [`languages.yml`](https://raw.githubusercontent.com/github/linguist/master/lib/linguist/languages.yml). When adding languages, follow the structure of existing entries, and use the language name defined by GitHub
in [`languages.yml`](https://raw.githubusercontent.com/github/linguist/master/lib/linguist/languages.yml).
Of course, PRs adding new languages are welcome and appreciated. Please add a test for your language in order for your PR to be accepted. See [Contributing](#contributing--issues). Of course, PRs adding new languages are welcome and appreciated. Please add a test for your language in order for your
PR to be accepted. See [Contributing](#contributing--issues).
## Setup ## Setup
@ -190,7 +198,7 @@ Create a `workflow.yml` file in your `.github/workflows` directory like:
```yml ```yml
name: "Run TODO to Issue" name: "Run TODO to Issue"
on: ["push"] on: [ "push" ]
jobs: jobs:
build: build:
runs-on: "ubuntu-latest" runs-on: "ubuntu-latest"
@ -200,24 +208,41 @@ jobs:
uses: "alstr/todo-to-issue-action@v4" uses: "alstr/todo-to-issue-action@v4"
``` ```
See [Github's workflow syntax](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions) for further details on this file. See [Github's workflow syntax](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions) for
further details on this file.
The workflow file takes the following optional inputs: The workflow file takes the following optional inputs:
| Input | Required | Description | | Parameter | Required | Description |
| ---------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |-----------------|----------|------------------------------------------------------------------------------------------------------------------------------------|
| `CLOSE_ISSUES` | No | Optional boolean input that specifies whether to attempt to close an issue when a TODO is removed. Default: `true`. | | REPO | False | The path to the repository where the action will be used, e.g., 'alstr/my-repo' (automatically set) |
| `AUTO_P` | No | Optional boolean input that specifies whether to format each line in multiline TODOs as a new paragraph. Default: `true`. | | BEFORE | False | The SHA of the last pushed commit (automatically set) |
| `IGNORE` | No | Optional string input that provides comma-delimited regular expressions that match files in the repo that we should not scan for TODOs. By default, we will scan all files. | | COMMITS | False | An array of commit objects describing the pushed commits |
| `AUTO_ASSIGN` | No | Optional boolean input that specifies whether to assign the newly created issue to the user who triggered the action. If users are manually assigned to an issue, this setting is ignored. Default: `false`. | | DIFF_URL | False | The URL to use to get the diff (automatically set) |
| `ISSUE_TEMPLATE` | No | You can override the default issue template by providing your own here. Markdown is supported, and you can inject the issue title, body, code URL and snippet. Example: `"This is my issue title: **{{ title }}**\n\nThis is my issue body: **{{ body }}**\n\nThis is my code URL: **{{ url }}**\n\nThis is my snippet:\n\n{{ snippet }}"` | | SHA | False | The SHA of the latest commit (automatically set) |
| `IDENTIFIERS` | No | A list of dictionaries specifying the identifiers for the action to recognise. `TODO` is the default, but you can override this here, and specify default labels to be applied when creating issues for each identifier. JSON string must be valid with double quoted keys/values and itself single-quoted (or double-quoted and escaped). Example: `'[{"name": "TODO", "labels": ["help wanted"]}, {"name": "FIXME", "labels": ["bug"]}]'` (`labels` should be an empty list if no default labels are wanted) | | TOKEN | False | The GitHub access token to allow us to retrieve, create and update issues (automatically set) |
| LABEL | False | The label that will be used to identify TODO comments (deprecated) |
| COMMENT_MARKER | False | The marker used to signify a line comment in your code (deprecated) |
| CLOSE_ISSUES | False | Optional input that specifies whether to attempt to close an issue when a TODO is removed |
| AUTO_P | False | For multiline TODOs, format each line as a new paragraph when creating the issue |
| PROJECTS_SECRET | False | Encrypted secret corresponding to your personal access token (do not enter the actual secret) |
| USER_PROJECTS | False | Default user projects |
| ORG_PROJECTS | False | Default organization projects |
| IGNORE | False | A collection of comma-delimited regular expressions that match files that should be ignored when searching for TODOs |
| AUTO_ASSIGN | False | Automatically assign new issues to the user who triggered the action |
| ACTOR | False | The username of the person who triggered the action |
| ISSUE_TEMPLATE | False | The template used to format new issues |
| IDENTIFIERS | False | Dictionary of custom identifiers |
| GITHUB_URL | False | Base URL of GitHub API |
| ESCAPE | False | Escape all special Markdown characters |
| LANGUAGES | False | A collection of comma-delimited URLs or local paths starting from the current working directory of the action for custom languages |
| NO_STANDARD | False | Exclude loading the default 'syntax.json' and 'language.yml' files from the repository |
These can be specified using `with` parameter in the workflow file, as below: These can be specified using `with` parameter in the workflow file, as below:
```yml ```yml
name: "Run TODO to Issue" name: "Run TODO to Issue"
on: ["push"] on: [ "push" ]
jobs: jobs:
build: build:
runs-on: "ubuntu-latest" runs-on: "ubuntu-latest"
@ -231,10 +256,86 @@ jobs:
### Considerations ### Considerations
- TODOs are found by analysing the difference between the new commit and its previous one (i.e., the diff). That means that if this action is implemented during development, any existing TODOs will not be detected. For them to be detected, you would have to remove them, commit, put them back, and commit again, or [run the action manually](#running-the-action-manually). - TODOs are found by analysing the difference between the new commit and its previous one (i.e., the diff). That means
that if this action is implemented during development, any existing TODOs will not be detected. For them to be
detected, you would have to remove them, commit, put them back, and commit again,
or [run the action manually](#running-the-action-manually).
- Should you change the TODO text, this will currently create a new issue. - Should you change the TODO text, this will currently create a new issue.
- Closing TODOs is still somewhat experimental. - Closing TODOs is still somewhat experimental.
## Custom Languages
If you want to add or overwrite language detections that are not currently supported, you can add them manually using the `LANGUAGES` input.
Just create a file that contains an array with languages, each having the following properties:
| Property | Type | Description |
|------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------|
| language | string | The unique name of the language |
| extensions | string[] | A list of file extensions for the custom language |
| markers | object[] | A list of objects (see example below) to declare the comment markers. Make sure to escape all special Markdown characters with a double backslash. |
For example, here is a language declaration file for Java:
```json
[
{
"language": "Java",
"extensions": [
".java"
],
"markers": [
{
"type": "line",
"pattern": "//"
},
{
"type": "block",
"pattern": {
"start": "/\\*",
"end": "\\*/"
}
}
]
}
]
```
Next, add the file to the `LANGUAGES` property in your workflow YAML file. Please note that if multiple paths are provided, the last path specified will take precedence over any previous ones:
**Using a Local File:**
```yaml
name: "Run TODO to Issue"
on: [ "push" ]
jobs:
build:
runs-on: "ubuntu-latest"
steps:
- uses: "actions/checkout@v3"
- name: "TODO to Issue"
uses: "alstr/todo-to-issue-action@v4"
with:
LANGUAGES: "path/to/my/file.json"
```
**Using a File from HTTP(s):**
```yaml
name: "Run TODO to Issue"
on: [ "push" ]
jobs:
build:
runs-on: "ubuntu-latest"
steps:
- uses: "actions/checkout@v3"
- name: "TODO to Issue"
uses: "alstr/todo-to-issue-action@v4"
with:
LANGUAGES: "http://myserver.com/path/to/my/file.json"
```
This will configure the action to use your custom language file for detecting TODO comments.
## Running the action manually ## Running the action manually
There may be circumstances where you want the action to run for a particular commit(s) already pushed. There may be circumstances where you want the action to run for a particular commit(s) already pushed.
@ -267,7 +368,8 @@ jobs:
Head to the Actions section of your repo, select the workflow and then 'Run workflow'. Head to the Actions section of your repo, select the workflow and then 'Run workflow'.
You can run the workflow for a single commit by entering the commit SHA in the first box. In this case, the action will compare the commit to the one directly before it. You can run the workflow for a single commit by entering the commit SHA in the first box. In this case, the action will
compare the commit to the one directly before it.
You can also compare a broader range of commits. For that, also enter the 'from'/base commit SHA in the second box. You can also compare a broader range of commits. For that, also enter the 'from'/base commit SHA in the second box.
@ -276,19 +378,24 @@ You can also compare a broader range of commits. For that, also enter the 'from'
### No issues have been created ### No issues have been created
- Make sure your file language is in `syntax.json`. - Make sure your file language is in `syntax.json`.
- The action will not recognise existing TODOs that have already been pushed, unless you [run the action manually](#running-the-action-manually). - The action will not recognise existing TODOs that have already been pushed, unless
- If a similar TODO appears in the diff as both an addition and deletion, it is assumed to have been moved, so is ignored. you [run the action manually](#running-the-action-manually).
- If your workflow is executed but no issue is generated, check your repo permissions by navigating to `Settings -> Actions (General) -> Workflow permissions` and enable "Read and write permissions". - If a similar TODO appears in the diff as both an addition and deletion, it is assumed to have been moved, so is
ignored.
- If your workflow is executed but no issue is generated, check your repo permissions by navigating
to `Settings -> Actions (General) -> Workflow permissions` and enable "Read and write permissions".
### Multiple issues have been created ### Multiple issues have been created
Issues are created whenever the action runs and finds a newly added TODO in the diff. Rebasing may cause a TODO to show up in a diff multiple times. This is an acknowledged issue, but you may have some luck by adjusting your workflow file. Issues are created whenever the action runs and finds a newly added TODO in the diff. Rebasing may cause a TODO to show
up in a diff multiple times. This is an acknowledged issue, but you may have some luck by adjusting your workflow file.
## Contributing & Issues ## Contributing & Issues
If you do encounter any problems, please file an issue or submit a PR. Everyone is welcome and encouraged to contribute. If you do encounter any problems, please file an issue or submit a PR. Everyone is welcome and encouraged to contribute.
**If submitting a request to add a new language, please ensure you add the appropriate tests covering your language. In the interests of stability, PRs without tests cannot be considered.** **If submitting a request to add a new language, please ensure you add the appropriate tests covering your language. In
the interests of stability, PRs without tests cannot be considered.**
## Running tests locally ## Running tests locally
@ -300,7 +407,8 @@ python -m unittest
## Customising ## Customising
If you want to fork this action to customise its behaviour, there are a few steps you should take to ensure your changes run: If you want to fork this action to customise its behaviour, there are a few steps you should take to ensure your changes
run:
- In `workflow.yml`, set `uses: ` to your action. - In `workflow.yml`, set `uses: ` to your action.
- In `action.yml`, set `image: ` to `Dockerfile`, rather than the prebuilt image. - In `action.yml`, set `image: ` to `Dockerfile`, rather than the prebuilt image.
@ -308,10 +416,15 @@ If you want to fork this action to customise its behaviour, there are a few step
## Thanks ## Thanks
The action was developed for the GitHub Hackathon. Whilst every effort is made to ensure it works, it comes with no guarantee. The action was developed for the GitHub Hackathon. Whilst every effort is made to ensure it works, it comes with no
guarantee.
Thanks to Jacob Tomlinson for [his handy overview of GitHub Actions](https://www.jacobtomlinson.co.uk/posts/2019/creating-github-actions-in-python/). Thanks to Jacob Tomlinson
for [his handy overview of GitHub Actions](https://www.jacobtomlinson.co.uk/posts/2019/creating-github-actions-in-python/).
Thanks to GitHub's [linguist repo](https://github.com/github/linguist/) for the [`languages.yml`](https://raw.githubusercontent.com/github/linguist/master/lib/linguist/languages.yml) file used by the app to look up file extensions and determine the correct highlighting to apply to code snippets. Thanks to GitHub's [linguist repo](https://github.com/github/linguist/) for
the [`languages.yml`](https://raw.githubusercontent.com/github/linguist/master/lib/linguist/languages.yml) file used by
the app to look up file extensions and determine the correct highlighting to apply to code snippets.
Thanks to all those who have [contributed](https://github.com/alstr/todo-to-issue-action/graphs/contributors) to the further development of this action. Thanks to all those who have [contributed](https://github.com/alstr/todo-to-issue-action/graphs/contributors) to the
further development of this action.

8
action.yml
View File

@ -80,3 +80,11 @@ inputs:
description: 'Escape all special Markdown characters' description: 'Escape all special Markdown characters'
required: false required: false
default: true default: true
LANGUAGES:
description: 'A collection of comma-delimited URLs or local paths starting from the current working directory of the action for custom languages'
required: false
default: ''
NO_STANDARD:
description: 'Exclude loading the default ''syntax.json'' and ''language.yml'' files from the repository'
required: false
default: false

60
main.py
View File

@ -301,7 +301,6 @@ class TodoParser(object):
def __init__(self): def __init__(self):
# Determine if the Issues should be escaped. # Determine if the Issues should be escaped.
self.should_escape = os.getenv('INPUT_ESCAPE', 'true') == 'true' self.should_escape = os.getenv('INPUT_ESCAPE', 'true') == 'true'
# Load any custom identifiers, otherwise use the default. # Load any custom identifiers, otherwise use the default.
custom_identifiers = os.getenv('INPUT_IDENTIFIERS') custom_identifiers = os.getenv('INPUT_IDENTIFIERS')
self.identifiers = ['TODO'] self.identifiers = ['TODO']
@ -319,6 +318,8 @@ class TodoParser(object):
self.languages_dict = None self.languages_dict = None
# Check if the standard collections should be loaded
if os.getenv('INPUT_NO_STANDARD', 'false') != 'true':
# Load the languages data for ascertaining file types. # Load the languages data for ascertaining file types.
languages_url = 'https://raw.githubusercontent.com/github/linguist/master/lib/linguist/languages.yml' languages_url = 'https://raw.githubusercontent.com/github/linguist/master/lib/linguist/languages.yml'
languages_request = requests.get(url=languages_url) languages_request = requests.get(url=languages_url)
@ -336,6 +337,63 @@ class TodoParser(object):
self.syntax_dict = syntax_request.json() self.syntax_dict = syntax_request.json()
else: else:
raise Exception('Cannot retrieve syntax data. Operation will abort.') raise Exception('Cannot retrieve syntax data. Operation will abort.')
else:
self.syntax_dict = []
self.languages_dict = {}
custom_languages = os.getenv('INPUT_LANGUAGES', '')
if custom_languages != '':
# Load all custom languages
for path in custom_languages.split(','):
try:
# Decide if the path is a url or local file
if path.startswith('http'):
languages_request = requests.get(path)
if languages_request.status_code != 200:
print('Cannot retrieve custom language file. (\''+path+'\')')
continue
data = languages_request.json()
else:
path = os.path.join(os.getcwd(), path)
if not os.path.exists(path) or not os.path.isfile(path):
print('Cannot retrieve custom language file. (\''+path+'\')')
continue
f = open(path)
data = json.load(f)
# Iterate through the definitions
for lang in data:
# Add/Replace the language definition
self.languages_dict[lang['language']] = {}
self.languages_dict[lang['language']]['type'] = ''
self.languages_dict[lang['language']]['color'] = ''
self.languages_dict[lang['language']]['extensions'] = lang['extensions']
self.languages_dict[lang['language']]['source'] = ''
self.languages_dict[lang['language']]['ace_mode'] = ''
self.languages_dict[lang['language']]['language_id'] = 0
# Check if a syntax with the language name already exists
counter = 0
exists = False
for syntax in self.syntax_dict:
if syntax['language'] == lang['language']:
exists = True
break
counter = counter + 1
if exists:
# When the syntax exists it will be popped out of the list
self.syntax_dict.pop(counter)
# And be replaced with the new syntax definition
self.syntax_dict.append({
'language': lang['language'],
'markers': lang['markers']
})
except:
print('An error occurred in the custom language file (\''+path+'\')')
print('Please check the file, or if it represents undefined behavior, create an issue at \'https://github.com/alstr/todo-to-issue-action/issues\'')
# noinspection PyTypeChecker # noinspection PyTypeChecker
def parse(self, diff_file): def parse(self, diff_file):

40
tests/custom_languages.json Normal file
View File

@ -0,0 +1,40 @@
[
{
"language": "ILS",
"extensions": [
".ils"
],
"markers": [
{
"type": "line",
"pattern": "//"
},
{
"type": "block",
"pattern": {
"start": "/\\*",
"end": "\\*/"
}
}
]
},
{
"language": "Java",
"extensions": [
".java2"
],
"markers": [
{
"type": "line",
"pattern": "////"
},
{
"type": "block",
"pattern": {
"start": "+=",
"end": "=+"
}
}
]
}
]

47
tests/test_todo_parser.py
View File

@ -93,6 +93,7 @@ class NewIssueTests(unittest.TestCase):
def test_xaml_issues(self): def test_xaml_issues(self):
self.assertEqual(count_issues_for_file_type(self.raw_issues, 'xml'), 2) self.assertEqual(count_issues_for_file_type(self.raw_issues, 'xml'), 2)
class ClosedIssueTests(unittest.TestCase): class ClosedIssueTests(unittest.TestCase):
# Check for removed TODOs across the files specified. # Check for removed TODOs across the files specified.
def setUp(self): def setUp(self):
@ -173,6 +174,7 @@ class ClosedIssueTests(unittest.TestCase):
def test_xaml_issues(self): def test_xaml_issues(self):
self.assertEqual(count_issues_for_file_type(self.raw_issues, 'xml'), 2) self.assertEqual(count_issues_for_file_type(self.raw_issues, 'xml'), 2)
class IgnorePatternTests(unittest.TestCase): class IgnorePatternTests(unittest.TestCase):
def test_single_ignore(self): def test_single_ignore(self):
@ -205,6 +207,7 @@ class IgnorePatternTests(unittest.TestCase):
self.assertEqual(count_issues_for_file_type(self.raw_issues, 'ruby'), 5) self.assertEqual(count_issues_for_file_type(self.raw_issues, 'ruby'), 5)
os.environ['INPUT_IGNORE'] = '' os.environ['INPUT_IGNORE'] = ''
class EscapeMarkdownTest(unittest.TestCase): class EscapeMarkdownTest(unittest.TestCase):
def test_simple_escape(self): def test_simple_escape(self):
os.environ['INPUT_ESCAPE'] = 'true' os.environ['INPUT_ESCAPE'] = 'true'
@ -227,3 +230,47 @@ class EscapeMarkdownTest(unittest.TestCase):
self.assertEqual(issue.body[0], '\\# Another title') self.assertEqual(issue.body[0], '\\# Another title')
self.assertEqual(issue.body[1], '\\<AnotherTag\\>') self.assertEqual(issue.body[1], '\\<AnotherTag\\>')
class customLanguageTest(unittest.TestCase):
def test_custom_lang_load(self):
os.environ['INPUT_LANGUAGES'] = 'tests/custom_languages.json'
parser = TodoParser()
# Test if the custom language ILS is actually loaded into the system
self.assertIsNotNone(parser.languages_dict['ILS'])
self.assertEqual(self.count_syntax(parser, 'ILS'), 1)
def test_custom_lang_not_dupplicate(self):
os.environ['INPUT_LANGUAGES'] = 'tests/custom_languages.json'
parser = TodoParser()
# Test if a custom language can overwrite the rules of an existing one
self.assertEqual(self.count_syntax(parser, 'Java'), 1)
for syntax in parser.syntax_dict:
if syntax['language'] == 'Java':
self.assertEqual(len(syntax['markers']), 2)
self.assertEqual(syntax['markers'][0]['pattern'], "////")
self.assertEqual(syntax['markers'][1]['pattern']['start'], '+=')
self.assertEqual(syntax['markers'][1]['pattern']['end'], '=+')
break
self.assertIsNotNone(parser.languages_dict['Java'])
self.assertEqual(len(parser.languages_dict['Java']['extensions']), 1)
self.assertEqual(parser.languages_dict['Java']['extensions'][0], ".java2")
def test_url_load(self):
os.environ['INPUT_LANGUAGES'] = 'https://raw.githubusercontent.com/alstr/todo-to-issue-action/blob/master/tests/custom_languages.json'
os.environ['INPUT_NO_STANDARD'] = 'true'
parser = TodoParser()
self.assertEqual(len(parser.languages_dict), 2)
self.assertEqual(len(parser.syntax_dict), 2)
@staticmethod
def count_syntax(parser: TodoParser, name: str):
counter = 0
for syntax in parser.syntax_dict:
if syntax['language'] == name:
counter = counter + 1
return counter