commit 901b21c555fa9820393b31ee54ade8cee915c895 · pyrox.dev/nixpkgs

+6 -5

doc/languages-frameworks/vim.section.md

···

       212
       212
        
       

     

       213
       213
        
       ## Adding new plugins to nixpkgs {#adding-new-plugins-to-nixpkgs}

     

       214
       214
        
       

     

       215
       215
       -
       Nix expressions for Vim plugins are stored in [pkgs/applications/editors/vim/plugins](https://github.com/NixOS/nixpkgs/tree/master/pkgs/applications/editors/vim/plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`./update.py`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/update.py). This creates a [generated.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/vim-plugin-names).

     

       215
       215
       +
       Nix expressions for Vim plugins are stored in [pkgs/applications/editors/vim/plugins](https://github.com/NixOS/nixpkgs/tree/master/pkgs/applications/editors/vim/plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`nix-shell -p vimPluginsUpdater --run vim-plugins-updater`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/updater.nix). This creates a [generated.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/vim-plugin-names).

     

       216
       216
        
       

     

       217
       217
       -
       After running `./update.py`, if nvim-treesitter received an update, also run [`nvim-treesitter/update.py`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/update.py) to update the tree sitter grammars for `nvim-treesitter`.

     

       217
       217
       +
       After running the updater, if nvim-treesitter received an update, also run [`nvim-treesitter/update.py`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/update.py) to update the tree sitter grammars for `nvim-treesitter`.

     

       218
       218
        
       

     

       219
       219
        
       Some plugins require overrides in order to function properly. Overrides are placed in [overrides.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/overrides.nix). Overrides are most often required when a plugin requires some dependencies, or extra steps are required during the build process. For example `deoplete-fish` requires both `deoplete-nvim` and `vim-fish`, and so the following override was added:

     

       220
       220
        
       

     
···

       241
       241
        
       Alternatively, set the number of processes to a lower count to avoid rate-limiting.

     

       242
       242
        
       

     

       243
       243
        
       ```sh

     

       244
       244
       -
       ./pkgs/applications/editors/vim/plugins/update.py --proc 1

     

       244
       244
       +
       

     

       245
       245
       +
       nix-shell -p vimPluginsUpdater --run 'vim-plugins-updater --proc 1'

     

       245
       246
        
       ```

     

       246
       247
        
       

     

       247
       248
        
       ## How to maintain an out-of-tree overlay of vim plugins ? {#vim-out-of-tree-overlays}

     
···

       250
       251
        
       plugin list:

     

       251
       252
        
       

     

       252
       253
        
       ```

     

       253
       253
       -
       pkgs/applications/editors/vim/plugins/update.py -i vim-plugin-names -o generated.nix --no-commit

     

       254
       254
       +
       nix-shell -p vimPluginsUpdater --run vim-plugins-updater -i vim-plugin-names -o generated.nix --no-commit

     

       254
       255
        
       ```

     

       255
       256
        
       

     

       256
       257
        
       with the contents of `vim-plugin-names` being for example:

     
···

       264
       265
        
       

     

       265
       266
        
       ```nix

     

       266
       267
        
       myVimPlugins = pkgs.vimPlugins.extend (

     

       267
       267
       -
         (pkgs.callPackage generated.nix {})

     

       268
       268
       +
         (pkgs.callPackage ./generated.nix {})

     

       268
       269
        
       );

     

       269
       270
        
       ```

     

       270
       271

+29 -11

maintainers/scripts/pluginupdate.py

···

       321
       321
        
           return plugins

     

       322
       322
        
       

     

       323
       323
        
       

     

       324
       324
       -
       def run_nix_expr(expr):

     

       325
       325
       -
           with CleanEnvironment() as nix_path:

     

       324
       324
       +
       

     

       325
       325
       +
       def run_nix_expr(expr, nixpkgs: str):

     

       326
       326
       +
           '''

     

       327
       327
       +
           :param expr nix expression to fetch current plugins

     

       328
       328
       +
           :param nixpkgs Path towards a nixpkgs checkout

     

       329
       329
       +
           '''

     

       330
       330
       +
           # local_pkgs = str(Path(__file__).parent.parent.parent)

     

       331
       331
       +
           with CleanEnvironment(nixpkgs) as nix_path:

     

       326
       332
        
               cmd = [

     

       327
       333
        
                   "nix",

     

       328
       334
        
                   "eval",

     
···

       396
       402
        
               """CSV spec"""

     

       397
       403
        
               print("the update member function should be overriden in subclasses")

     

       398
       404
        
       

     

       399
       399
       -
           def get_current_plugins(self) -> List[Plugin]:

     

       405
       405
       +
           def get_current_plugins(self, nixpkgs) -> List[Plugin]:

     

       400
       406
        
               """To fill the cache"""

     

       401
       401
       -
               data = run_nix_expr(self.get_plugins)

     

       407
       407
       +
               data = run_nix_expr(self.get_plugins, nixpkgs)

     

       402
       408
        
               plugins = []

     

       403
       409
        
               for name, attr in data.items():

     

       404
       410
        
                   p = Plugin(name, attr["rev"], attr["submodules"], attr["sha256"])

     
···

       414
       420
        
               raise NotImplementedError()

     

       415
       421
        
       

     

       416
       422
        
           def get_update(self, input_file: str, outfile: str, config: FetchConfig):

     

       417
       417
       -
               cache: Cache = Cache(self.get_current_plugins(), self.cache_file)

     

       423
       423
       +
               cache: Cache = Cache(self.get_current_plugins(self.nixpkgs), self.cache_file)

     

       418
       424
        
               _prefetch = functools.partial(prefetch, cache=cache)

     

       419
       425
        
       

     

       420
       426
        
               def update() -> dict:

     
···

       454
       460
        
                   ),

     

       455
       461
        
               )

     

       456
       462
        
               common.add_argument(

     

       463
       463
       +
                   "--nixpkgs",

     

       464
       464
       +
                   type=str,

     

       465
       465
       +
                   default=os.getcwd(),

     

       466
       466
       +
                   help="Adjust log level",

     

       467
       467
       +
               )

     

       468
       468
       +
               common.add_argument(

     

       457
       469
        
                   "--input-names",

     

       458
       470
        
                   "-i",

     

       459
       471
        
                   dest="input_file",

     
···

       541
       553
        
               command = args.command or "update"

     

       542
       554
        
               log.setLevel(LOG_LEVELS[args.debug])

     

       543
       555
        
               log.info("Chose to run command: %s", command)

     

       556
       556
       +
               self.nixpkgs = args.nixpkgs

     

       544
       557
        
       

     

       545
       545
       -
               if not args.no_commit:

     

       546
       546
       -
                   self.nixpkgs_repo = git.Repo(self.root, search_parent_directories=True)

     

       558
       558
       +
               self.nixpkgs_repo = git.Repo(args.nixpkgs, search_parent_directories=True)

     

       547
       559
        
       

     

       548
       560
        
               getattr(self, command)(args)

     

       549
       561
        
       

     

       550
       562
        
       

     

       551
       563
        
       class CleanEnvironment(object):

     

       564
       564
       +
           def __init__(self, nixpkgs):

     

       565
       565
       +
               self.local_pkgs = nixpkgs

     

       566
       566
       +
       

     

       552
       567
        
           def __enter__(self) -> str:

     

       553
       553
       -
               self.old_environ = os.environ.copy()

     

       568
       568
       +
               """

     

       554
       569
        
               local_pkgs = str(Path(__file__).parent.parent.parent)

     

       570
       570
       +
               """

     

       571
       571
       +
               self.old_environ = os.environ.copy()

     

       555
       572
        
               self.empty_config = NamedTemporaryFile()

     

       556
       573
        
               self.empty_config.write(b"{}")

     

       557
       574
        
               self.empty_config.flush()

     

       558
       558
       -
               os.environ["NIXPKGS_CONFIG"] = self.empty_config.name

     

       559
       559
       -
               return f"localpkgs={local_pkgs}"

     

       575
       575
       +
               # os.environ["NIXPKGS_CONFIG"] = self.empty_config.name

     

       576
       576
       +
               return f"localpkgs={self.local_pkgs}"

     

       560
       577
        
       

     

       561
       578
        
           def __exit__(self, exc_type: Any, exc_value: Any, traceback: Any) -> None:

     

       562
       579
        
               os.environ.update(self.old_environ)

     
···

       758
       775
        
       

     

       759
       776
        
       

     

       760
       777
        
       def update_plugins(editor: Editor, args):

     

       761
       761
       -
           """The main entry function of this module. All input arguments are grouped in the `Editor`."""

     

       778
       778
       +
           """The main entry function of this module.

     

       779
       779
       +
           All input arguments are grouped in the `Editor`."""

     

       762
       780
        
       

     

       763
       781
        
           log.info("Start updating plugins")

     

       764
       782
        
           fetch_config = FetchConfig(args.proc, args.github_token)

+53 -45

maintainers/scripts/update-luarocks-packages

···

       2
       2
        
       #!nix-shell update-luarocks-shell.nix -i python3

     

       3
       3
        
       

     

       4
       4
        
       # format:

     

       5
       5
       -
       # $ nix run nixpkgs.python3Packages.black -c black update.py

     

       5
       5
       +
       # $ nix run nixpkgs#python3Packages.black -- update.py

     

       6
       6
        
       # type-check:

     

       7
       7
       -
       # $ nix run nixpkgs.python3Packages.mypy -c mypy update.py

     

       7
       7
       +
       # $ nix run nixpkgs#python3Packages.mypy -- update.py

     

       8
       8
        
       # linted:

     

       9
       9
       -
       # $ nix run nixpkgs.python3Packages.flake8 -c flake8 --ignore E501,E265,E402 update.py

     

       9
       9
       +
       # $ nix run nixpkgs#python3Packages.flake8 -- --ignore E501,E265,E402 update.py

     

       10
       10
        
       

     

       11
       11
        
       import inspect

     

       12
       12
        
       import os

     
···

       25
       25
        
       log = logging.getLogger()

     

       26
       26
        
       log.addHandler(logging.StreamHandler())

     

       27
       27
        
       

     

       28
       28
       -
       ROOT = Path(os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))).parent.parent # type: ignore

     

       28
       28
       +
       ROOT = Path(os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))).parent.parent  # type: ignore

     

       29
       29
        
       import pluginupdate

     

       30
       30
        
       from pluginupdate import update_plugins, FetchConfig, CleanEnvironment

     

       31
       31
        
       

     

       32
       32
       -
       PKG_LIST="maintainers/scripts/luarocks-packages.csv"

     

       33
       33
       -
       TMP_FILE="$(mktemp)"

     

       34
       34
       -
       GENERATED_NIXFILE="pkgs/development/lua-modules/generated-packages.nix"

     

       35
       35
       -
       LUAROCKS_CONFIG="maintainers/scripts/luarocks-config.lua"

     

       32
       32
       +
       PKG_LIST = "maintainers/scripts/luarocks-packages.csv"

     

       33
       33
       +
       TMP_FILE = "$(mktemp)"

     

       34
       34
       +
       GENERATED_NIXFILE = "pkgs/development/lua-modules/generated-packages.nix"

     

       35
       35
       +
       LUAROCKS_CONFIG = "maintainers/scripts/luarocks-config.lua"

     

       36
       36
        
       

     

       37
       37
        
       HEADER = """/* {GENERATED_NIXFILE} is an auto-generated file -- DO NOT EDIT!

     

       38
       38
        
       Regenerate it with:

     
···

       40
       40
        
       

     

       41
       41
        
       You can customize the generated packages in pkgs/development/lua-modules/overrides.nix

     

       42
       42
        
       */

     

       43
       43
       -
       """.format(GENERATED_NIXFILE=GENERATED_NIXFILE)

     

       43
       43
       +
       """.format(

     

       44
       44
       +
           GENERATED_NIXFILE=GENERATED_NIXFILE

     

       45
       45
       +
       )

     

       44
       46
        
       

     

       45
       45
       -
       FOOTER="""

     

       47
       47
       +
       FOOTER = """

     

       46
       48
        
       }

     

       47
       49
        
       /* GENERATED - do not edit this file */

     

       48
       50
        
       """

     

       49
       51
        
       

     

       52
       52
       +
       

     

       50
       53
        
       @dataclass

     

       51
       54
        
       class LuaPlugin:

     

       52
       55
        
           name: str

     

       53
       53
       -
           '''Name of the plugin, as seen on luarocks.org'''

     

       56
       56
       +
           """Name of the plugin, as seen on luarocks.org"""

     

       54
       57
        
           src: str

     

       55
       55
       -
           '''address to the git repository'''

     

       58
       58
       +
           """address to the git repository"""

     

       56
       59
        
           ref: Optional[str]

     

       57
       57
       -
           '''git reference (branch name/tag)'''

     

       60
       60
       +
           """git reference (branch name/tag)"""

     

       58
       61
        
           version: Optional[str]

     

       59
       59
       -
           '''Set it to pin a package '''

     

       62
       62
       +
           """Set it to pin a package """

     

       60
       63
        
           server: Optional[str]

     

       61
       61
       -
           '''luarocks.org registers packages under different manifests.

     

       64
       64
       +
           """luarocks.org registers packages under different manifests.

     

       62
       65
        
           Its value can be 'http://luarocks.org/dev'

     

       63
       63
       -
           '''

     

       66
       66
       +
           """

     

       64
       67
        
           luaversion: Optional[str]

     

       65
       65
       -
           '''Attribue of the lua interpreter if a package is available only for a specific lua version'''

     

       68
       68
       +
           """Attribue of the lua interpreter if a package is available only for a specific lua version"""

     

       66
       69
        
           maintainers: Optional[str]

     

       67
       67
       -
           ''' Optional string listing maintainers separated by spaces'''

     

       70
       70
       +
           """ Optional string listing maintainers separated by spaces"""

     

       68
       71
        
       

     

       69
       72
        
           @property

     

       70
       73
        
           def normalized_name(self) -> str:

     

       71
       74
        
               return self.name.replace(".", "-")

     

       75
       75
       +
       

     

       72
       76
        
       

     

       73
       77
        
       # rename Editor to LangUpdate/ EcosystemUpdater

     

       74
       78
        
       class LuaEditor(pluginupdate.Editor):

     
···

       77
       81
        
       

     

       78
       82
        
           def load_plugin_spec(self, input_file) -> List[LuaPlugin]:

     

       79
       83
        
               luaPackages = []

     

       80
       80
       -
               csvfilename=input_file

     

       84
       84
       +
               csvfilename = input_file

     

       81
       85
        
               log.info("Loading package descriptions from %s", csvfilename)

     

       82
       86
        
       

     

       83
       83
       -
               with open(csvfilename, newline='') as csvfile:

     

       84
       84
       -
                   reader = csv.DictReader(csvfile,)

     

       87
       87
       +
               with open(csvfilename, newline="") as csvfile:

     

       88
       88
       +
                   reader = csv.DictReader(

     

       89
       89
       +
                       csvfile,

     

       90
       90
       +
                   )

     

       85
       91
        
                   for row in reader:

     

       86
       92
        
                       # name,server,version,luaversion,maintainers

     

       87
       93
        
                       plugin = LuaPlugin(**row)

     
···

       91
       97
        
           def update(self, args):

     

       92
       98
        
               update_plugins(self, args)

     

       93
       99
        
       

     

       94
       94
       -
           def generate_nix(

     

       95
       95
       -
               self,

     

       96
       96
       -
               results: List[Tuple[LuaPlugin, str]],

     

       97
       97
       -
               outfilename: str

     

       98
       98
       -
               ):

     

       99
       99
       -
       

     

       100
       100
       +
           def generate_nix(self, results: List[Tuple[LuaPlugin, str]], outfilename: str):

     

       100
       101
        
               with tempfile.NamedTemporaryFile("w+") as f:

     

       101
       102
        
                   f.write(HEADER)

     

       102
       103
        
                   header2 = textwrap.dedent(

     

       103
       103
       -
                   # header2 = inspect.cleandoc(

     

       104
       104
       -
                   """

     

       104
       104
       +
                       # header2 = inspect.cleandoc(

     

       105
       105
       +
                       """

     

       105
       106
        
                       { self, stdenv, lib, fetchurl, fetchgit, callPackage, ... } @ args:

     

       106
       107
        
                       final: prev:

     

       107
       108
        
                       {

     

       108
       108
       -
                   """)

     

       109
       109
       +
                   """

     

       110
       110
       +
                   )

     

       109
       111
        
                   f.write(header2)

     

       110
       110
       -
                   for (plugin, nix_expr) in results:

     

       112
       112
       +
                   for plugin, nix_expr in results:

     

       111
       113
        
                       f.write(f"{plugin.normalized_name} = {nix_expr}")

     

       112
       114
        
                   f.write(FOOTER)

     

       113
       115
        
                   f.flush()

     
···

       156
       158
        
               #         luaPackages.append(plugin)

     

       157
       159
        
               pass

     

       158
       160
        
       

     

       161
       161
       +
       

     

       159
       162
        
       def generate_pkg_nix(plug: LuaPlugin):

     

       160
       160
       -
           '''

     

       163
       163
       +
           """

     

       161
       164
        
           Generate nix expression for a luarocks package

     

       162
       165
        
           Our cache key associates "p.name-p.version" to its rockspec

     

       163
       163
       -
           '''

     

       166
       166
       +
           """

     

       164
       167
        
           log.debug("Generating nix expression for %s", plug.name)

     

       165
       168
        
           custom_env = os.environ.copy()

     

       166
       166
       -
           custom_env['LUAROCKS_CONFIG'] = LUAROCKS_CONFIG

     

       169
       169
       +
           custom_env["LUAROCKS_CONFIG"] = LUAROCKS_CONFIG

     

       167
       170
        
       

     

       168
       171
        
           # we add --dev else luarocks wont find all the "scm" (=dev) versions of the

     

       169
       172
        
           # packages

     

       170
       170
       -
       	# , "--dev"

     

       171
       171
       -
           cmd = [ "luarocks", "nix" ]

     

       173
       173
       +
           # , "--dev"

     

       174
       174
       +
           cmd = ["luarocks", "nix"]

     

       172
       175
        
       

     

       173
       176
        
           if plug.maintainers:

     

       174
       177
        
               cmd.append(f"--maintainers={plug.maintainers}")

     
···

       176
       179
        
           # if plug.server == "src":

     

       177
       180
        
           if plug.src != "":

     

       178
       181
        
               if plug.src is None:

     

       179
       179
       -
                   msg = "src must be set when 'version' is set to \"src\" for package %s" % plug.name

     

       182
       182
       +
                   msg = (

     

       183
       183
       +
                       "src must be set when 'version' is set to \"src\" for package %s"

     

       184
       184
       +
                       % plug.name

     

       185
       185
       +
                   )

     

       180
       186
        
                   log.error(msg)

     

       181
       187
        
                   raise RuntimeError(msg)

     

       182
       188
        
               log.debug("Updating from source %s", plug.src)

     
···

       185
       191
        
           else:

     

       186
       192
        
               cmd.append(plug.name)

     

       187
       193
        
               if plug.version and plug.version != "src":

     

       188
       188
       -
       

     

       189
       194
        
                   cmd.append(plug.version)

     

       190
       195
        
       

     

       191
       196
        
           if plug.server != "src" and plug.server:

     
···

       194
       199
        
           if plug.luaversion:

     

       195
       200
        
               cmd.append(f"--lua-version={plug.luaversion}")

     

       196
       201
        
       

     

       197
       197
       -
           log.debug("running %s", ' '.join(cmd))

     

       202
       202
       +
           log.debug("running %s", " ".join(cmd))

     

       198
       203
        
       

     

       199
       204
        
           output = subprocess.check_output(cmd, env=custom_env, text=True)

     

       200
       205
        
           output = "callPackage(" + output.strip() + ") {};\n\n"

     

       201
       206
        
           return (plug, output)

     

       202
       207
        
       

     

       203
       203
       -
       def main():

     

       204
       208
        
       

     

       205
       205
       -
           editor = LuaEditor("lua", ROOT, '',

     

       206
       206
       -
               default_in = ROOT.joinpath(PKG_LIST),

     

       207
       207
       -
               default_out = ROOT.joinpath(GENERATED_NIXFILE)

     

       208
       208
       -
               )

     

       209
       209
       +
       def main():

     

       210
       210
       +
           editor = LuaEditor(

     

       211
       211
       +
               "lua",

     

       212
       212
       +
               ROOT,

     

       213
       213
       +
               "",

     

       214
       214
       +
               default_in=ROOT.joinpath(PKG_LIST),

     

       215
       215
       +
               default_out=ROOT.joinpath(GENERATED_NIXFILE),

     

       216
       216
       +
           )

     

       209
       217
        
       

     

       210
       218
        
           editor.run()

     

       211
       219
        
       

     

       212
       212
       -
       if __name__ == "__main__":

     

       213
       220
        
       

     

       221
       221
       +
       if __name__ == "__main__":

     

       214
       222
        
           main()

     

       215
       223
        
       

     

       216
       224
        
       #  vim: set ft=python noet fdm=manual fenc=utf-8 ff=unix sts=0 sw=4 ts=4 :

+16 -12

pkgs/applications/editors/vim/plugins/nvim-treesitter/update.py

···

       7
       7
        
       from os import environ

     

       8
       8
        
       from os.path import dirname, join

     

       9
       9
        
       

     

       10
       10
       -
       lockfile = json.load(open(join(environ["NVIM_TREESITTER"], "lockfile.json")))

     

       11
       11
       -
       

     

       12
       10
        
       configs = json.loads(

     

       13
       11
        
           subprocess.check_output(

     

       14
       12
        
               [

     
···

       58
       56
        
           return generated

     

       59
       57
        
       

     

       60
       58
        
       

     

       61
       61
       -
       generated_file = """# generated by pkgs/applications/editors/vim/plugins/nvim-treesitter/update.py

     

       59
       59
       +
       def update_grammars(lockfile: str):

     

       62
       60
        
       

     

       63
       63
       -
       { buildGrammar, """

     

       61
       61
       +
           generated_file = """# generated by pkgs/applications/editors/vim/plugins/nvim-treesitter/update.py

     

       64
       62
        
       

     

       65
       65
       -
       generated_file += subprocess.check_output(["nurl", "-Ls", ", "], text=True)

     

       63
       63
       +
           { buildGrammar, """

     

       66
       64
        
       

     

       67
       67
       -
       generated_file += """ }:

     

       65
       65
       +
           generated_file += subprocess.check_output(["nurl", "-Ls", ", "], text=True)

     

       68
       66
        
       

     

       69
       69
       -
       {

     

       70
       70
       -
       """

     

       67
       67
       +
           generated_file += """ }:

     

       71
       68
        
       

     

       72
       72
       -
       for generated in ThreadPoolExecutor().map(generate_grammar, lockfile.items()):

     

       73
       73
       -
           generated_file += generated

     

       69
       69
       +
           {

     

       70
       70
       +
           """

     

       71
       71
       +
           for generated in ThreadPoolExecutor().map(generate_grammar, lockfile.items()):

     

       72
       72
       +
               generated_file += generated

     

       73
       73
       +
               generated_file += "}\n"

     

       74
       74
       +
               generated_file += "}\n"

     

       74
       75
        
       

     

       75
       75
       -
       generated_file += "}\n"

     

       76
       76
       +
           open(join(dirname(__file__), "generated.nix"), "w").write(generated_file)

     

       77
       77
       +
       

     

       76
       78
        
       

     

       77
       77
       -
       open(join(dirname(__file__), "generated.nix"), "w").write(generated_file)

     

       79
       79
       +
       if __name__ == "__main__":

     

       80
       80
       +
           # TODO add lockfile

     

       81
       81
       +
           update_grammars()

-16

pkgs/applications/editors/vim/plugins/update-shell.nix

···

       1
       1
       -
       { pkgs ? import ../../../../.. { } }:

     

       2
       2
       -
       

     

       3
       3
       -
       # Ideally, pkgs points to default.nix file of Nixpkgs official tree

     

       4
       4
       -
       with pkgs;

     

       5
       5
       -
       let

     

       6
       6
       -
         pyEnv = python3.withPackages (ps: [ ps.gitpython ]);

     

       7
       7
       -
       in

     

       8
       8
       -
       

     

       9
       9
       -
       mkShell {

     

       10
       10
       -
         packages = [

     

       11
       11
       -
           bash

     

       12
       12
       -
           pyEnv

     

       13
       13
       -
           nix

     

       14
       14
       -
           nix-prefetch-scripts

     

       15
       15
       -
         ];

     

       16
       16
       -
       }

+44 -45

pkgs/applications/editors/vim/plugins/update.py

···

       1
       1
       -
       #!/usr/bin/env nix-shell

     

       2
       2
       -
       #!nix-shell update-shell.nix -i python3

     

       3
       3
       -
       

     

       1
       1
       +
       #!/usr/bin/env python

     

       4
       2
        
       

     

       3
       3
       +
       # run with:

     

       4
       4
       +
       # $ nix run .\#vimPluginsUpdater

     

       5
       5
        
       # format:

     

       6
       6
       -
       # $ nix run nixpkgs.python3Packages.black -c black update.py

     

       6
       6
       +
       # $ nix run nixpkgs#python3Packages.black -- update.py

     

       7
       7
        
       # type-check:

     

       8
       8
       -
       # $ nix run nixpkgs.python3Packages.mypy -c mypy update.py

     

       8
       8
       +
       # $ nix run nixpkgs#python3Packages.mypy -- update.py

     

       9
       9
        
       # linted:

     

       10
       10
       -
       # $ nix run nixpkgs.python3Packages.flake8 -c flake8 --ignore E501,E265,E402 update.py

     

       10
       10
       +
       # $ nix run nixpkgs#python3Packages.flake8 -- --ignore E501,E265,E402 update.py

     

       11
       11
        
       

     

       12
       12
       -
       # If you see `HTTP Error 429: too many requests` errors while running this script,

     

       13
       13
       -
       # refer to:

     

       12
       12
       +
       # If you see `HTTP Error 429: too many requests` errors while running this

     

       13
       13
       +
       # script, refer to:

     

       14
       14
        
       #

     

       15
       15
        
       # https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/vim.section.md#updating-plugins-in-nixpkgs-updating-plugins-in-nixpkgs

     

       16
       16
        
       #

     

       17
       17
       -
       # (or the equivalent file /doc/languages-frameworks/vim.section.md from Nixpkgs master tree).

     

       17
       17
       +
       # (or the equivalent file /doc/languages-frameworks/vim.section.md

     

       18
       18
       +
       # from Nixpkgs master tree).

     

       18
       19
        
       #

     

       19
       20
        
       

     

       20
       21
        
       import inspect

     

       21
       22
        
       import os

     

       22
       22
       -
       import sys

     

       23
       23
        
       import logging

     

       24
       24
       -
       import subprocess

     

       25
       24
        
       import textwrap

     

       25
       25
       +
       import json

     

       26
       26
        
       from typing import List, Tuple

     

       27
       27
        
       from pathlib import Path

     

       28
       28
        
       

     

       29
       29
       -
       import git

     

       30
       29
        
       

     

       31
       30
        
       log = logging.getLogger()

     

       32
       31
        
       

     
···

       37
       36
        
       

     

       38
       37
        
       # Import plugin update library from maintainers/scripts/pluginupdate.py

     

       39
       38
        
       ROOT = Path(os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe()))))

     

       40
       40
       -
       # Ideally, ROOT.(parent^5) points to root of Nixpkgs official tree

     

       41
       41
       -
       sys.path.insert(

     

       42
       42
       -
           0, os.path.join(ROOT.parent.parent.parent.parent.parent, "maintainers", "scripts")

     

       43
       43
       -
       )

     

       44
       39
        
       import pluginupdate

     

       40
       40
       +
       import importlib

     

       45
       41
        
       from pluginupdate import run_nix_expr, PluginDesc

     

       46
       46
       -
       nvim_treesitter = importlib.import_module("treesitter")

     

       42
       42
       +
       from treesitter import update_grammars

     

       47
       43
        
       

     

       48
       48
       -
       

     

       49
       49
       -
       GET_PLUGINS_LUA = """

     

       50
       50
       -
       with import <localpkgs> {};

     

       51
       51
       -
       lib.attrNames lua51Packages"""

     

       52
       44
        
       

     

       53
       45
        
       HEADER = (

     

       54
       46
        
           "# GENERATED by ./pkgs/applications/editors/vim/plugins/update.py. Do not edit!"

     

       55
       47
        
       )

     

       56
       48
        
       

     

       57
       57
       -
       

     

       58
       58
       -
       def isNeovimPlugin(plug: pluginupdate.Plugin) -> bool:

     

       59
       59
       -
           """

     

       60
       60
       -
           Whether it's a neovim-only plugin

     

       61
       61
       -
           We can check if it's available in lua packages

     

       62
       62
       -
           """

     

       63
       63
       -
           global luaPlugins

     

       64
       64
       -
           if plug.normalized_name in luaPlugins:

     

       65
       65
       -
               log.debug("%s is a neovim plugin", plug)

     

       66
       66
       -
               return True

     

       67
       67
       -
           return False

     

       49
       49
       +
       NVIM_TREESITTER_GENERATED_NIX = \

     

       50
       50
       +
           "pkgs/applications/editors/vim/plugins/nvim-treesitter/generated.nix"

     

       68
       51
        
       

     

       69
       52
        
       

     

       70
       53
        
       class VimEditor(pluginupdate.Editor):

     
···

       75
       58
        
           ):

     

       76
       59
        
               sorted_plugins = sorted(plugins, key=lambda v: v[0].name.lower())

     

       77
       60
        
               nvim_treesitter_rev = pluginupdate.run_nix_expr(

     

       78
       78
       -
                   "(import <localpkgs> { }).vimPlugins.nvim-treesitter.src.rev"

     

       61
       61
       +
                   "(import <localpkgs> { }).vimPlugins.nvim-treesitter.src.rev",

     

       62
       62
       +
                   self.nixpkgs

     

       79
       63
        
               )

     

       80
       64
        
       

     

       81
       65
        
               with open(outfile, "w+") as f:

     
···

       94
       78
        
                       content = self.plugin2nix(pdesc, plugin)

     

       95
       79
        
                       f.write(content)

     

       96
       80
        
                       if (

     

       97
       97
       -
                           plugin.name == "nvim-treesitter"

     

       98
       98
       -
                           and plugin.commit != nvim_treesitter_rev

     

       81
       81
       +
                           plugin.name == "nvim-treesitter" and plugin.commit != nvim_treesitter_rev

     

       99
       82
        
                       ):

     

       100
       83
        
                           self.nvim_treesitter_updated = True

     

       101
       84
        
                   f.write("\n}\n")

     

       102
       85
        
               print(f"updated {outfile}")

     

       103
       86
        
       

     

       104
       87
        
           def plugin2nix(self, pdesc: PluginDesc, plugin: pluginupdate.Plugin) -> str:

     

       88
       88
       +
               GET_PLUGINS_LUA = """

     

       89
       89
       +
               with import <localpkgs> {};

     

       90
       90
       +
               lib.attrNames lua51Packages"""

     

       91
       91
       +
               luaPlugins = run_nix_expr(GET_PLUGINS_LUA, self.nixpkgs)

     

       92
       92
       +
       

     

       105
       93
        
               repo = pdesc.repo

     

       106
       106
       -
               isNeovim = isNeovimPlugin(plugin)

     

       94
       94
       +
       

     

       95
       95
       +
               def _isNeovimPlugin(plug: pluginupdate.Plugin) -> bool:

     

       96
       96
       +
                   """

     

       97
       97
       +
                   Whether it's a neovim-only plugin

     

       98
       98
       +
                   We can check if it's available in lua packages

     

       99
       99
       +
                   """

     

       100
       100
       +
                   # global luaPlugins

     

       101
       101
       +
                   if plug.normalized_name in luaPlugins:

     

       102
       102
       +
                       log.debug("%s is a neovim plugin", plug)

     

       103
       103
       +
                       return True

     

       104
       104
       +
                   return False

     

       105
       105
       +
       

     

       106
       106
       +
               isNeovim = _isNeovimPlugin(plugin)

     

       107
       107
        
       

     

       108
       108
        
               content = f"  {plugin.normalized_name} = "

     

       109
       109
        
               src_nix = repo.as_nix(plugin)

     
···

       129
       129
        
               if self.nvim_treesitter_updated:

     

       130
       130
        
                   print("updating nvim-treesitter grammars")

     

       131
       131
        
                   nvim_treesitter_dir = ROOT.joinpath("nvim-treesitter")

     

       132
       132
       -
                   # subprocess.check_call([nvim_treesitter_dir.joinpath("update.py")])

     

       133
       133
       -
                   nvim_treesitter.update_grammars()

     

       132
       132
       +
                   lockfile = json.load(open(args.nixpkgs.join(NVIM_TREESITTER_GENERATED_FILE, "lockfile.json")))

     

       133
       133
       +
       

     

       134
       134
       +
                   nvim_treesitter.update_grammars(lockfile)

     

       134
       135
        
       

     

       135
       136
        
                   if self.nixpkgs_repo:

     

       136
       137
        
                       index = self.nixpkgs_repo.index

     

       137
       138
        
                       for diff in index.diff(None):

     

       138
       138
       -
                           if (

     

       139
       139
       -
                               diff.a_path

     

       140
       140
       -
                               == "pkgs/applications/editors/vim/plugins/nvim-treesitter/generated.nix"

     

       141
       141
       -
                           ):

     

       139
       139
       +
                           if diff.a_path == NVIM_TREESITTER_GENERATED_NIX:

     

       142
       140
        
                               msg = "vimPlugins.nvim-treesitter: update grammars"

     

       143
       141
        
                               print(f"committing to nixpkgs: {msg}")

     

       144
       142
        
                               index.add([str(nvim_treesitter_dir.joinpath("generated.nix"))])

     
···

       148
       146
        
       

     

       149
       147
        
       

     

       150
       148
        
       def main():

     

       149
       149
       +
       

     

       151
       150
        
           global luaPlugins

     

       152
       152
       -
           luaPlugins = run_nix_expr(GET_PLUGINS_LUA)

     

       153
       151
        
       

     

       154
       154
       -
           with open(f"{ROOT}/get-plugins.nix") as f:

     

       152
       152
       +
           log.debug(f"Loading from {ROOT}/../get-plugins.nix")

     

       153
       153
       +
           with open(f"{ROOT}/../get-plugins.nix") as f:

     

       155
       154
        
               GET_PLUGINS = f.read()

     

       156
       156
       -
           editor = VimEditor("vim", ROOT, GET_PLUGINS)

     

       155
       155
       +
           editor = VimEditor("vim", Path("pkgs/applications/editors/vim/plugins"), GET_PLUGINS)

     

       157
       156
        
           editor.run()

     

       158
       157
        
       

     

       159
       158

+47

pkgs/applications/editors/vim/plugins/updater.nix

···

       1
       1
       +
       { buildPythonApplication

     

       2
       2
       +
       , nix

     

       3
       3
       +
       , makeWrapper

     

       4
       4
       +
       , python3Packages

     

       5
       5
       +
       , lib

     

       6
       6
       +
       

     

       7
       7
       +
       # optional

     

       8
       8
       +
       , vimPlugins

     

       9
       9
       +
       , neovim

     

       10
       10
       +
       }:

     

       11
       11
       +
       let

     

       12
       12
       +
         my_neovim = neovim.override {

     

       13
       13
       +
           configure.packages.all.start = [ vimPlugins.nvim-treesitter ];

     

       14
       14
       +
         };

     

       15
       15
       +
       

     

       16
       16
       +
       in

     

       17
       17
       +
       buildPythonApplication {

     

       18
       18
       +
         format = "other";

     

       19
       19
       +
         pname = "vim-plugins-updater";

     

       20
       20
       +
         version = "0.1";

     

       21
       21
       +
       

     

       22
       22
       +
         nativeBuildInputs = [

     

       23
       23
       +
           makeWrapper

     

       24
       24
       +
           python3Packages.wrapPython

     

       25
       25
       +
         ];

     

       26
       26
       +
       

     

       27
       27
       +
         pythonPath = [

     

       28
       28
       +
           python3Packages.gitpython

     

       29
       29
       +
         ];

     

       30
       30
       +
       

     

       31
       31
       +
         dontUnpack = true;

     

       32
       32
       +
       

     

       33
       33
       +
         installPhase = ''

     

       34
       34
       +
           mkdir -p $out/bin $out/lib

     

       35
       35
       +
           cp ${./update.py} $out/bin/vim-plugins-updater

     

       36
       36
       +
           cp ${./get-plugins.nix} $out/get-plugins.nix

     

       37
       37
       +
           cp ${./nvim-treesitter/update.py} $out/lib/treesitter.py

     

       38
       38
       +
           cp ${../../../../../maintainers/scripts/pluginupdate.py} $out/lib/pluginupdate.py

     

       39
       39
       +
       

     

       40
       40
       +
           # wrap python scripts

     

       41
       41
       +
           makeWrapperArgs+=( --prefix PATH : "${lib.makeBinPath [ nix my_neovim ]}" --prefix PYTHONPATH : "$out/lib" )

     

       42
       42
       +
           wrapPythonPrograms

     

       43
       43
       +
         '';

     

       44
       44
       +
       

     

       45
       45
       +
         meta.mainProgram = "vim-plugins-updater";

     

       46
       46
       +
       }

     

       47
       47
       +

pkgs/top-level/all-packages.nix

···

       126
       126
        
       

     

       127
       127
        
         common-updater-scripts = callPackage ../common-updater/scripts.nix { };

     

       128
       128
        
       

     

       129
       129
       +
         # vimPluginsUpdater = callPackage ../applications/editors/vim/plugins/updater.nix {

     

       130
       130
       +
         #   inherit (writers) writePython3Bin;

     

       131
       131
       +
         # };

     

       132
       132
       +
         vimPluginsUpdater = callPackage ../applications/editors/vim/plugins/updater.nix {

     

       133
       133
       +
           inherit (python3Packages) buildPythonApplication ;

     

       134
       134
       +
         };

     

       135
       135
       +
       

     

       129
       136
        
         genericUpdater = callPackage ../common-updater/generic-updater.nix { };

     

       130
       137
        
       

     

       131
       138
        
         _experimental-update-script-combinators = callPackage ../common-updater/combinators.nix { };