Add PageFind beta V2 (Full Text Search across Site)#2857
Add PageFind beta V2 (Full Text Search across Site)#2857MoshiMoshiMochi merged 42 commits intoMarkBind:masterfrom
Conversation
This added support for configuring Pagefind search indexing via site.json is aimed towards allowing users to declare glob patterns to customize what pages in which directory should be indexed by pagefind.
Initially, I felt that including the root_selector option and force_language option would be useful for pagefind configuration options. But perhaps for now it isn't the most pressing issue so I'm going to exlcude it from the current implementation of pagefind.
This helps resolve an issue with generate() test case within
Site.functional.test.ts. Namely, because Jest's cannot intercept
eval('import("pagefind")') runtime dynamic import used by pagefind.
Hence, for now at least, the error handling would catch this error and
allow the site generation to continue normally.
Extracted processResult into its own helper function. Added rel="noopener noreferrer" attribute for improved security and privacy when being redirected to pagefind's github repo.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2857 +/- ##
==========================================
+ Coverage 70.41% 70.88% +0.47%
==========================================
Files 129 131 +2
Lines 6952 7083 +131
Branches 1578 1660 +82
==========================================
+ Hits 4895 5021 +126
- Misses 1957 1962 +5
Partials 100 100 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Hi guys, sorry for the wait, marking this as ready to review! As mentioned in the PR description, while I believe there are still plenty of things that we can work on to enhancing the Pagefind feature in the future. Namely, we could take a look at
We can probably make separate issues for these things. The current implementation is far from perfect (e.g. a lot of css can be improved, clearly still an issue when using at different browser sizes, also a lot of z level issues with the navbar & nav components). So let me know of what yall think of the current implementation, anything to improve, anything pressing to fix, etc. @MarkBind/active-3281-members |
There was a problem hiding this comment.
Pull request overview
Adds Pagefind-based full-text search to MarkBind as a beta feature, including build-time indexing, asset injection into generated pages, a new Vue <search /> modal UI, and supporting configuration/docs/test updates.
Changes:
- Introduce a new Vue
Searchcomponent (Pagefind UI modal) and styling assets. - Generate and inject Pagefind search assets during site generation, with
site.jsonsupport forpagefind.exclude_selectorsandpagefind.glob. - Update unit/functional tests, expected HTML outputs, and tooling ignores to accommodate generated Pagefind artifacts.
Reviewed changes
Copilot reviewed 128 out of 130 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/vue-components/src/pagefindSearchBar/assets/search.css | Adds global CSS for the Pagefind search modal styling. |
| packages/vue-components/src/pagefindSearchBar/Search.vue | New Pagefind-backed search modal component + keyboard handling and UI overrides. |
| packages/vue-components/src/pagefindSearchBar/LogoPagefind.vue | Adds Pagefind logo SVG for the modal footer. |
| packages/vue-components/src/index.js | Registers the new Search component for <search /> usage. |
| packages/vue-components/src/tests/Search.spec.js | Adds unit tests for the new Search component behavior. |
| packages/core/test/unit/utils/data.ts | Adds factories/helpers for mocking Pagefind and generating site.json with Pagefind config. |
| packages/core/test/unit/Site/SiteGenerationManager.test.ts | Adds unit tests for Pagefind glob normalization/validation and indexing behavior. |
| packages/core/src/Site/SitePagesManager.ts | Injects Pagefind UI CSS/JS asset paths into page rendering assets when search is enabled. |
| packages/core/src/Site/SiteGenerationManager.ts | Adds Pagefind indexing during site generation and glob filtering support. |
| packages/core/src/Site/SiteConfig.ts | Extends site config to include optional pagefind configuration. |
| packages/core/src/Page/page.njk | Injects Pagefind UI stylesheet/script into generated pages when available. |
| packages/core/src/Page/PageConfig.ts | Adds pagefindCss/pagefindJs to PageAssets type. |
| packages/core/package.json | Adds pagefind dependency for build-time indexing. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/UserGuide.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/QuickStart.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/Features.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/FAQ.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/team/johndoe.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/team/AboutUs.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/TracingCode.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Testing.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/SettingUp.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Requirements.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Implementation.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Documentation.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/DeveloperGuide.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/DevOps.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Design.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Configuration.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_portfolio/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_minimal/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_default/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic3b.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic3a.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic2.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic1.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_templates/test_default/expected/404.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_table_plugin/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_special_tags/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/test_folder/extra_3.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/test_folder/extra_2.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/test_folder/extra_1.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic3b.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic3a.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic2.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic1.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/about.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/README.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/Page-1.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/Home.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/404.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic3b.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic3a.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic2.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic1.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/about.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/_Sidebar.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/_Footer.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/Page-1.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/Home.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/404.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site_algolia_plugin/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/test_md_fragment.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testWeb3FormPlugin.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testVariableContainsInclude.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testTree.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testTooltipSpacing.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testThumbnails.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testSourceContainScript.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testSingleAltFrontMatter.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPopovers.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPopoverTrigger.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPlantUML.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPanelsClosingTransition.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPanels.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPanelMarkdownParsing.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPageNavWithoutTitleAndNavHeadings.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPageNavWithOnlyTitle.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPageNavTarget.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPageNavPrint.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testPageNav.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testOcticonInPage.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testNunjucksPathResolving.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testModals.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testMermaid.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testMath.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testMaterialIconsInPage.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testList.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testLinks.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testLayoutsWithAltFrontMatter.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testLayoutsOverrideWithAltFrontmatter.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testLayoutsOverride.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testLayouts.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testIncludePluginsRendered.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testIncludeMultipleModals.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testIncludeBoilerplate.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testImages.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testIconsInSiteLayout.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testHr.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testGlyphiconInPage.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testFontAwesomeInPage.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testExternalScripts.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testEmptyFrontmatter.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testEmptyAltFrontMatter.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testDates.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testCodeBlocks.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testCenterText.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testBootstrapIconInPage.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testAntiFOUCStyles.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testAnnotate.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testAnchorGeneration.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testAltFrontMatterParsing.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/testAltFrontMatterInvalidKeyValue.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/sub_site/testNunjucksPathResolving.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/sub_site/nested_sub_site/testNunjucksPathResolving.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/sub_site/nested_sub_site/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/sub_site/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/test_site/expected/bugs/index.html | Updates expected HTML to include Pagefind UI assets. |
| packages/cli/test/functional/testUtil/compare.ts | Adds ignore support for generated Pagefind outputs in functional comparisons. |
| packages/cli/test/functional/test.ts | Uses new compare() signature to ignore generated pagefind/ directories. |
| package-lock.json | Locks Pagefind and platform-specific optional dependencies. |
| docs/userGuide/makingTheSiteSearchable.md | Documents Pagefind beta usage and new site.json options. |
| .stylelintrc.js | Adds stylelint override for Pagefind UI class naming patterns. |
| .gitignore | Ignores generated Pagefind directories and fragments. |
| .eslintrc.js | Adds ESLint ignore patterns for generated/build output directories. |
| .eslintignore | Ignores generated Pagefind JS under functional test outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
packages/vue-components/src/pagefindSearchBar/assets/search.css
Outdated
Show resolved
Hide resolved
Harjun751
left a comment
There was a problem hiding this comment.
Huge effort, and it definitely paid off!
Requesting some changes/clarifications for the
- debug.log file
- empty glob parsing
- windows directory safety check
- spawned watcher
issues, but other than that it's all minor nits. After the above I think we'll be in a good state.
Now indexSiteWithPagefind returns a boolean which is then determines if there is a need to inject the pagefindcss & pagefindJs.
Since isValidGlobPattern is only used by normalizeGlobPattern, changed it to private access. Now normalization of pattern happens solely in normalizeGlobPattern and validation is purely a validation method. Better separation of concerns. Also resolved a bug where the 404s required pages to be created before knowing if indexing succeeds by defaulting index success to true and only if it fails does the flag get set to false.
|
Good first step introducing pagefind into MarkBind @MoshiMoshiMochi . The build-time integration (Node API, The main concern is in
This is workable for a first cut, but the cause-and-effect is opaque, where future contributors would need to understand both the PagefindUI internals and overrides to make changes safely. It also loads pagefind-ui.js (84.6KB) + pagefind-ui.css (14.5KB) on every page, when we only need pagefind.js (33.8KB). The long-term direction should be:
Suggest tracking this as follow-up issues:
I think it is okay to approve as a foundation if these follow-ups are tracked, since the build-time indexing is correct and the search does work end-to-end, just wanted to clarify on your architectural direction for the component moving forward! @MoshiMoshiMochi |
Harjun751
left a comment
There was a problem hiding this comment.
Changes I requested are addressed 👍
|
@MoshiMoshiMochi Each PR must have a SEMVER impact label, please remember to label the PR properly. |

What is the purpose of this pull request?
Overview of changes:
Initial version of pagefind (based on #2771)
Original Search Issue is #205
No test cases yet.
Anything you'd like to highlight/discuss:
Search.vuecomponent<search />that renders the pagefind search modal.Heres how the modal currently looks vs Algolia's (CS2103 site)
Current implementation of pagefind.

Algolia's styling on CS2103 website

From the 2 images above, I believe that Algolia's styling is notably more structured in how it represents the search results and subresults, notably on how they group the search results.
The blog we used as reference for the styling formats the pagefindresults by processing the search results. More information on how to access the specific search results information can be found here.
pagefindconfiguration options withinsite.jsonto enable more flexible filtering options. Users can declareexclude_selectorsoption to exclude specific elements from being searchable.globoption to limit which pages are indexed by Pagefind.{ "pagefind": { "exclude_selectors": [".algolia-no-index", "[class*='algolia-no-index']"] "glob": ["**/devGuide/**", "**/userGuide/**"] } }exclude_selectors: tells Pagefind to exclude any element with thealgolia-no-indexclass from the searchable results. It works the same way as declaringdata-pagefind-ignore.glob: Pagefind will only index pages that are found insidedevGuideanduserGuidedirectories (including subdirectories).All in all, theres still a lot more things we can do to improve this pagefind feature (like page weighting, ranking, sorting etc). But this is all I've managed to conjure up thus far.
Let me know what yall think of these updates to the pagefind feature (there might be better ways to filter pages, this is just from my interpretation when reading through the pagefind docs). Let me know if I missed out on anything in particular or should be focusing my efforts more towards.
Testing instructions:
Proposed commit message: (wrap lines at 72 characters)
Add base Pagefind with updated UI & extra filtering functionalities
Checklist: ☑️
Reviewer checklist:
Indicate the SEMVER impact of the PR:
At the end of the review, please label the PR with the appropriate label:
r.Major,r.Minor,r.Patch.Breaking change release note preparation (if applicable):