Strings are not required to be UTF-8. Go source code is required to be UTF-8. There is a complex path between the two.
In short, there are three kinds of strings. They are:
- the substring of the source that lexes into a string literal.
- a string literal.
- a value of type string.
Only the first is required to be UTF-8. The second is required to be written in UTF-8, but its contents are interpreted various ways and may encode arbitrary bytes. The third can contain any bytes at all.
Try this on:
var s string = "\xFF語"
Source substring: "\xFF語", UTF-8 encoded. The data:
22
5c
78
46
46
e8
aa
9e
22
String literal: \xFF語 (between the quotes). The data:
5c
78
46
46
e8
aa
9e
The string value (unprintable; this is a UTF-8 stream). The data:
ff
e8
aa
9e
And for record, the characters (code points):
<erroneous byte FF, will appear as U+FFFD if you range over the string value>
語 U+8a9e
AVX512 Articles AssemblyPolicy Benchmarks Blogs Books BoundingResourceUse CSSStyleGuide ChromeOS CodeReview CodeReviewComments CodeReviewConcurrency CodeTools Comments CommitMessage CommonMistakes CompilerOptimizations Conferences Configuring GoLand for WebAssembly Contributing to gopls CoreDumpDebugging Courses CreatingSubRepository CustomPprofProfiles Darwin DashboardBuilders Deprecated DesignDocuments Diagnostics Download build farm failed logs and debugging DragonFly BSD ErrorValueFAQ Errors ExperienceReports FileTreeDocumentation FreeBSD FromXToGo Frozen Fuzzing trophy case GOPATH Gardening GcToolchainTricks GccgoCrossCompilation GerritAccess GerritBot GithubAccess Go 1.10 Release Party Go 1.6 release party Go 1.8 Release Party Go Community Slides Go Release Cycle Go2 Go2ErrorHandlingFeedback Go2ErrorValuesFeedback Go2GenericsFeedback GoArm GoForCPPProgrammers GoGenerateTools GoGetProxyConfig GoGetTools GoMips GoStrings GoTalks GoUserGroups GoUsers Gomote Gopher HandlingIssues Home HostedContinuousIntegration How to ask for help HowToAsk IDEsAndTextEditorPlugins InstallFromSource InstallTroubleshooting InterfaceSlice InvalidFlag Iota Learn LearnConcurrency LearnErrorHandling LearnServerProgramming LearnTesting Linux LinuxKernelSignalVectorBug Livestreams LockOSThread MacOS12BSDThreadRegisterIssue MethodSets MinimumRequirements MinorReleases Mobile Modules MutexOrChannel NativeClient NetBSD NewSpeakers NoMeToo NoPlusOne NonEnglish OpenBSD PackagePublishing PanicAndRecover PerfDashboard Performance Plan9 Podcasts PortingPolicy PriorDiscussion Projects Proposals ProviderIntegration Questions Quiet Weeks Range RateLimiting ResearchPapers Resolving Problems From Modified Module Path Resources for slog SQLDrivers SQLInterface Screencasts SettingGOPATH SignalHandling SimultaneousAssignment SliceTricks SlowBots Solaris Spectre Spelling Style SuccessStories Switch TableDrivenTests TargetSpecific TestComments TestFailures Timeouts Training Ubuntu Watchflakes WebAccessibilityResourcesAndTips Well known struct tags WhyGo Windows WindowsBuild WindowsCrossCompiling WindowsDLLs X Repositories _Footer cgo golang tools gopherbot gopls integrator FAQ gopls heapdump13 heapdump14 heapdump15 through heapdump17 heapdump15